Tag: teaching metacognition

How a Metacognitive Card Game was Born

by John Draeger, SUNY Buffalo State

I have developed a variety of metacognitive exercises over the years, but I never thought I’d develop a card game. The nerd in me likes the fact that a metacognitive card game now exists. While I want to tell you about how the game came about, I am also writing because the story includes an important metacognitive moment.

Promoting metacognition in traditional educational settingsimage showing four example metacognition cards

Most of my metacognitive exercises have been designed for classroom settings where students are asked to become aware of their various learning processes (such as reading, writing, ethical reasoning) and their experience with their process (such as feeling stuck). If students can become aware of what’s working and what’s not, then they can better recognize where they need to make adjustments and find ways to improve.

As a small part of a National Science Foundation (NSF) funded project, I developed metacognition exercises to support students doing course-based undergraduate research (EvaluateUR-CURE).[1] These efforts yielded a collection of twelve exercises that provide students with opportunities to build metacognitive habits as well as a guidebook for instructors (Metacognitive exercises and Guides). These exercises provide students with opportunities to practice metacognition in ways that support the broad elements of EvaluateUR-CURE (E-CURE for short).

Each metacognitive exercise helps students attend to various elements of their research process (such as reading for research, developing good research questions, managing projects, building resilience, and effectively communicating results). Once aware of their process, students can then decide whether they need to make adjustments to their process or perhaps seek out additional resources. The guidebook provides instructors and mentors with quick ways to integrate student metacognition into the conversations that they are already having in the classroom. While not specifically designed to support the version of the EvaluateUR method for students conducting independent research with a faculty mentor, the exercises developed for E-CURE can be very useful in a variety of settings.

Time for an adjustment?

My design challenge changed dramatically when working on another NSF funded project aimed at developing a variant of the method – called Evaluate-Compete – for students participating in engineering design competitions. The goal was to help students develop and become aware of academic and workforce skills through their involvement in these competitions. The NSF project initially focused on the MATE ROV competition where high school and college students build underwater vehicles to simulate solving various real-world marine challenges. Teams work all year on their vehicles and then compete in a series of regional competitions in hopes of qualifying for the World Championships. As with the course-based research projects, I was tasked with designing exercises to support metacognitive growth.

I quickly encountered a problem. Remotely Operated Vehicles (ROVs) are fun. I had the good fortune of attending the MATE ROV world championships in 2021 and 2022. It is easy to see why teams spend countless hours designing, testing, and tweaking their ROVs. It’s fun. Therein lies both secret to student engagement and flaw in my plan to use worksheets to promote metacognition.

Before, during, and after the competitions, the focus is on the vehicles, vehicle systems, and vehicle performance. The students creating these vehicles are learning all sort of things (and that’s ultimately the point), but it surely seems like it is all about the vehicles. In the previous NSF project, metacognitive exercises encouraged students to pause long enough to consider where they might need to make adjustments in their process and how they might take those lessons into the next learning context. Hands-on research was the secret to student engagement, but the classroom or the structured mentoring setting was the secret to students completing and discussing the metacognitive exercises.

Unlike those settings, students building ROVs were not always tied to formal classes and often work as part of a club, student organization, or a group of friends seeing if they were up to the challenge. The goal in designing the exercise was to promote skill building and reflection on their learning process as they design vehicles and solve problems. But why would anyone do metacognitive exercises when there is no grade attached, especially if you could be focusing on an ROV instead?

Why Cards?

It was time for me to take my own metacognitive medicine. Metacognition refers to an intentional awareness about a process. My process for designing metacognitive questions was going well enough and it worked in other settings, but my initial attempts to pitch worksheets to eager young engineers fell flat and it was quickly obvious that the strategy was never going to work.

Metacognition reminds us to make adjustments when a particular strategy isn’t working. My strategy was doomed. I needed an adjustment. If I could get students to pause and reflect on their process, then the prompts had a chance of helping them build better habits. But how to get them to pause long enough to engage the prompts? I needed a fun way to engage students in reflection about their learning that could be done anywhere, anytime, with any number of people. Because regular practice is important to habit formation, I further hoped that something fun could improve frequency of use and aid habit formation. I shifted strategies and designed a metacognitive card game.

Want to play?

The resulting metacognitive cards can be used at any time, in any order, or in any combination. They can be played by individuals or by a team. Each contains a series of “fun” prompts organized in three categories – problem-solving, persistence, and working with people. Ways to play can be found within each deck, including on planes, trains, at team meetings, and pizza parties. For example, the game can be played individually or in a group. When teams play, individuals can answer from the point of view of someone else in the group, pass a card to someone else for them to respond, or shift perspective by answering from the point of view of someone outside the group. Early feedback on the cards is positive and feedback collection is ongoing. I’ll have more to say future posts. For now, if YOU would like to play the Better upon reflection: building metacognitive habits ONE card at a time  card game, you can request a deck or print out your own deck. And if you play, I’d love to hear how it goes.

[1] Special thanks to the EvaluateUR-Method team – Jill Singer, Sean Fox, Daniel Weiler, Jill Zande, Emma Binder, Maureen Kahn, and Bridget Zimmerman.


Facilitating Metacognition in a First-Year Writing Course

By Dr. Gina Burkart, Clarke University

Metacognition and First-Year Students

The first year of college can be a difficult transition for students, as they often lack many of the basic skills to navigate the cognitive dissonance that happens that first year. Integrating tools, strategies, and assessments into first-year writing courses that foster self-reflection also offer students the opportunity to think about their own thinking. Students are often unable to “bridge the gap” of college because they don’t have the “meta-discourse” or “meta-awareness” to engage in the writing and discussions of the university (Gennrich & Dison, 2018, pp. 4-5). Essentially, it is important for first-year students to engage in what Flavell (1979) described as metacognition—thinking about what they are doing, how they are doing it, and why they are doing it, so they can make adjustments to accurately and effectively meet the needs of the situation, purpose, and audience (Victori, 1999).

This type of rhetorical analysis builds metacognitive practices integral to the first-year writing and cancartoon graphic of hands working on a computer with a paper and pens next to it (from Pixabay) guide students in discovering their own voice and learning how to use it in different academic disciplines. Within the first-year writing course, portfolio conferences are especially helpful in leading students through this type of metacognitive practice (Gencel, 2017; Alexiou & Parakeva, 2010; 2013; Farahian & Avazamani, 2018). However, course curriculum should be thoughtfully created so that students are first introduced to metacognition in the beginning of the semester and led through it repeatedly throughout the semester (Schraw, 1999).

Designing a First-Year Writing Course that Facilitates Metacognition

While self-reflection has always been part of the first-year writing courses I have taught, this year I introduced metacognition in the beginning of the semester and reinforced it throughout the semester as students engaged with the course theme – Motivation. I embedded my college-success curriculum 16 Weeks to College Success: The Mindful Student, Kendall Hunt into the course, which provided tools and strategies for the students to use as they learned college writing practices.

This approach was particularly helpful in a writing class because the metacognitive reflective activities reinforced writing as a tool for learning.  For example, during the first-week of the semester, students discussed the syllabus in groups and created learning plans for the course using a template. The template helped them pull out essential information and think through a personalized action plan to find success within the context of the course (see Learning Plan Template, Figure 1). For example, based on the course readings: How would they read? What strategies would they use? How much time would they devote to the reading? How often would they read? What resources would they use? Students turned in their learning plans for assigned points—and I read through the plans and made comments and suggestions. Students were told that they would update their plans over the course of the semester, which they would place in their portfolio to display growth over time.

Students also took two self-assessments, set goals based on their self-assessments, and created a time management and study plan to achieve their goals. Similar to the learning plans, these activities were also turned in for points and feedback and included in their final portfolios to demonstrate growth. One asked them to self-assess on a scale of 1-5 in key skills areas that impact success: Reading, Writing, Note-Taking, Time Management, Organization, Test-Taking, Oral Communication, Studying, and Motivation. This self-assessment also included questions in metacognitive reflection about the skills and how they relate to the course. The second self-assessment (included in the 16 Weeks textbook) was the LASSI —a nationally normed self-assessment on the key indicators of college success—provided quantitative data on a scale of 1-99. The LASSI dimensions were mapped to the skill areas for success to facilitate students’ goal setting using the goal setting chart (See Goal Setting Chart, Figure 2).

Throughout the semester, students completed skill and strategy activities from the 16 Weeks textbook supporting students’ goals and helping them complete course assignments. For example, when students were assigned their first reading assignments, the strategies for critical reading were also introduced and assigned. Students received points and feedback on their use of the reading strategies applied to the course readings, and demonstrated their application of the reading strategies through a reading journal. This reading journal was included in the portfolio (see Reading Journal template, Figure 3).

Students also were taught note-taking strategies so they could take more effective notes while watching assigned course videos on Growth Mindset, Grit and Emotional Courage. As a result, students were able to apply and learn new strategies while being exposed to key concepts related to identity, self-reflection, and metacognition. In fact, all of the course writing, speaking, and reading assignments also led the students to learn and think about topics that reinforce metacognition.

Portfolio Assessment

As portfolio assessment has been shown to positively affect metacognition and writing instruction (Farahian & Avarzmani, 2018; Alexiou & Parakva, 2010; 2013; Gencel, 2017), it seemed an appropriate culminating assessment for the course. The course curriculum built toward the portfolio throughout the semester in that the course assignments included the four steps suggested by Schaw (1998): 1) introduced and reinforced an awareness of metacognition 2) supported course learning and use of strategies 2) encouraged regulated learning 3) offered a setting that was rich with metacognition. Additionally, the course curriculum and final portfolio assessment conference included the three suggested variables of metacognitive knowledge: person, task, and strategic.

Students created a cover letter for their portfolios describing their growth in achieving the goals they set over the course of the semester. Specifically, they described new strategies, tools and resources they used and applied in the course and their other courses that helped them grow in their goal areas. The portfolio included artifacts demonstrating the application and growth. Additionally, students included their self-assessments and adjusted learning plans. Students also took the LASSI assessment as a post test in week 14 and were asked to include the pre and post-test assessments in the portfolios to compare their quantitative results and discuss growth and continued growth as part of the final conference (See Portfolio Rubric, Figure 4).

When meeting with students, students read their cover letters and talked me through their portfolios, showing me how they used and applied strategies and grew in their goal areas over the course of the semester. Part of the conversation included how they would continue to apply and/or adjust strategies, tools, and resources to continue the growth.

To reinforce metacognition and self-reflection, I had students score themselves with the portfolio and cover letter rubrics. I then scored them, and we discussed the scores. In all instances, students either scored themselves lower or the same as I scored them. They also appreciated hearing me discuss how I arrived at the scores and appreciated feedback that I had regarding their work.

In summary, students enjoyed the portfolio conferences and shared that they wished more professors used portfolios as assessments. They also shared that they enjoyed looking at their growth and putting the portfolio together. All students expressed a deeper understanding of self and expectations of college writing, reading, and learning. They also demonstrated an understanding of strategies and tools to use moving forward and gratitude for being given tools and strategies. LASSI scores demonstrated greatest growth in the skill areas of: Anxiety, Selecting Main Ideas, Self-Testing (Fall semester); Time Management, Concentration, Information Processing/Self-Testing (Spring semester). While the results reinforce that students show different areas of growth, students in both classes demonstrated highest areas of growth in reading (Selecting Main Ideas or Information Processing). Additionally, these skill areas (Time Management, Concentration, and Self-Testing) demonstrate the ability to self-regulate—suggesting that regular reinforcement of metacognition throughout the writing course and the portfolio assignment may have had a positive effect on growth and the acquisition of metacognitive practices (Shraw, 1999; Veenman, Van Hout-Wolters, & Afflerbach, 2006).

References

Alexiou, A., & Paraskeva, F. (2010). Enhancing self-regulated learning skills through the implementation of an e-portfolio tool. Procedia Social and Behavioral Sciences, 2, 3048–3054.

Burkart, G. (2023). 16 weeks to college success: The mindful student. Dubuque, Iowa: Kendall Hunt.

Farahian  M. & Avarzamanim, F.. (2018) The impact of portfolio on EFL learners’ metacognition and writing performance, Cogent Education, 5:1, 1450918, https://doi.org/10.1080/2331186X.2018.1450918 

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911. https://doi.org/10.1037/0003-066X.34.10.906

Gencel, I.E. (2017). The effect of portfolio assessments on metacognitive skills and on attitudes toward a course. Educational Sciences: Theory & Practice, 293-319.

Gennrich, T. & Dison, L. (2018). Voice matters: Students struggle to find voice. Reading & Writing, 9(1), 1-8.

Schraw, G. (1999). The effect of metacognitive knowledge on local and global monitoring. Contemporary Educational Psychology, 19, 143-154.

Veenman, M. V., Van Hout-Wolters, B. H., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and learning, 1, 3-14.

Victori, M. (1999). An analysis of writing knowledge in EFL composing: A case study of two effective and two less effective writers. System, 27(4), 537-555.

Weinstein, C., Palmer, D., & Acee, T. (2024). LASSI: Learning and study strategy inventory. https://www.collegelassi.com/lassi/index.html

 


Fostering Metacognition to Support Student Learning and Performance

This article by Julie Dangremond Stanton, Amanda J. Sebesta and John Dunlosky “outline the reasons metacognition is critical for learning and summarize relevant research … in … three main areas in which faculty can foster students’ metacognition: supporting student learning strategies (i.e., study skills), encouraging monitoring and control of learning, and promoting social metacognition during group work.” They then “distill insights from key papers into general recommendations for instruction, as well as a special list of four recommendations that instructors can implement in any course.”

CBE Life Sci Educ June 1, 2021 20:fe3

https://doi.org/10.1187/cbe.20-12-0289


Using Ungrading and Metacognition to Foster “Becoming a Learner”

by Matt Recla, PhD, Associate Director of University Foundations at Boise State

This is the 3rd post in the Guest Editor Series, Metacognition, Writing, and Well-Being, Edited by dawn shepherd, PhD, Ti Macklin, PhD, and Heidi Estrem, PhD

Becoming a Learner

When I started teaching a required first-year course years ago, faculty were recommended to include Matthew Sanders’ small text, Becoming a Learner. Though it seemed a distraction from the “real” content of my course, I dutifully added the text. It makes a simple, compelling argument that students should strive to be active learners rather than passive students, exposing common misconceptions about a college education and suggesting helpful corrections. I paired the text with a short assignment to craft three learning goals for the semester, including at least one for our course and at least one for their learning journey more broadly.

I was surprised by the overwhelmingly positive reactions from students. Though assigned at the beginning of the semester, in their reflections on the course months later students still made comments like the following: “I learned so much about myself and what to improve on.” “It really set the tone for the rest of the class.” “It really changed my perspective on how I view my college education.” A few even claimed it was the most valuable part of the course! Reflecting on their past learning experiences and considering concrete goals provided a tool to gain purchase on their educational journey.colorful silhouette of human head with colors exploding from top

I began to wonder, though, whether my teaching techniques and assignments throughout the rest of the course were in harmony with the message of becoming a learner. Sanders exhorts students to be creative and courageous in order to learn (14, 42). Was I helping students do that, or was I penalizing them if they took a risk? He encourages critical thinking and the interconnectedness of learning (15, 35). Was I providing opportunities to make those connections, to reflect on the impact of their learning? These reflections led me to further opportunities for student metacognition. I made two additional changes that, in offering students a greater sense of empowerment in their education, also hopefully contributes to their sense of well-being.

Ungrading

The first change was ungrading, which to my mind was the natural complement to a first-year required course that promotes taking charge of your education. (There are many different ways to ungrade; I was initially guided by Hacking Assessment, and have since benefitted from the edited volume, Ungrading.) I’ve landed for now on a system where students receive no grades until the end of the course. They receive significant feedback on each assignment (based on Mark Barnes’ SE2R feedback approach) from me or a teaching assistant and have unlimited opportunities to revise and resubmit their work. We meet individually with each one of our 100 students at midsemester to hear about their progress and tackle any ongoing challenges. We meet again at semester’s end, and students explain the grade they believe they’ve earned. At least nine times out of ten they assess themselves just as we (instructors) would. When there appear to be gaps in the student’s self-assessment, we have a slightly longer conversation to understand (and rarely, suggest possible corrections to) their rationale.

I have come to see ungrading as part of my own well-being as an educator, as it appropriately shares my responsibility for a student’s grade with them. They are well-positioned to evaluate their performance if I trust them to do so and let them practice. There is a learning curve, and it can at first be frustrating for students who (like myself as a student) are used to finding out “what the teacher wants.” If embraced, though, it encourages for most students more authentic engagement with their learning. Their reflections suggest this augments a feeling of ownership of their education.

Metacognitive Reflection

The second change I adopted is to have students write or record a brief metacognitive reflection along with every major assignment. (My first and last assignments are themselves reflections on their experience, so I don’t assign a reflection on their reflection. That gets confusing for everyone!) The prompt for this brief addendum asks students to think about successes and challenges, both internal and external. (I’ve lost track of the original source for this idea, but I’m grateful!) I show these four areas in a quadrant and invite students to respond to at least one prompt in each area:

 

Internal

External

Successes

●   What did I do to achieve success on this assignment?

●   What did I learn from this assessment (in terms of content, skills, and/or about myself)?

●   What parts of the assignment worked well for me? Why?

●   Where do I think I did best on the assignment or what portion am I particularly proud of?

●   Which assignment standards did I meet or exceed? Why do I think so?

Challenges

●   What challenges did I face while completing the assignment (outside the assignment itself)?

●   How did I overcome those challenges?

●   What do I plan to do differently next time as a result?

●   What parts of the assignment were most challenging for me to understand? Why?

●   How did I overcome those challenges?

●   Which assignment standards did I not meet? Why?

Students reflect honestly on their challenges and modestly on their successes. They already may do this internally as they complete their work, but taking the time to record it helps reinforce that intuitive reflection and reveals the interconnectedness of their learning. The reflections often provide helpful context for their work, which may be impacted by any number of factors. In most cases I can affirm their self-assessment and suggest other small shifts as needed. The opportunity for intentional, transparent reflection has induced some “aha!” moments. I’ve seen many students follow through with changes in their time management for future assignments or double-down on areas of skill that were uncovered in reflection, which, because self-generated rather than forced, increases their felt self-efficacy.

Teaching in a COVID (and post-COVID) world

Although I incorporated both of these practices before the global disruptions of the last couple years, I’ve found that both ungrading and metacognitive reflection lend themselves well to teaching in a world unmoored by a pandemic. In the fall semester of 2020 we could see the impacts of a dramatic disruption in students’ learning, transitioning from in-person to primarily or completely virtual. Those impacts have become more pronounced each year since. The flexible design of my course is fairly adaptable to student needs and abilities when they enter the course, and it means that their grade isn’t ruined because they miss something due to unforeseen circumstances.

As they complete assignments and reflect on their progress, I can see them wrestle with the challenges of my course while simultaneously managing their other courses and the numerous obligations of adulthood. When they reflect at the end of the semester and assign themselves a grade, I can see how they comprehensively assess what this small piece of their growth as learners has added up to. I am privileged to work with students with a variety of different experiences and perspectives, and if my classroom provides a space where they can reflect on where they are and continue the lifelong process of becoming learners, I feel that I’ve boosted their well-being and not hindered their journey.

References:

Blum, S. D. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press.

Sackstein, S. (2015). Hacking assessment: 10 ways to go gradeless in a traditional grades school. Times 10 Publications.

Sanders, M. L. (2018). Becoming a learner: Realizing the opportunity of education. Macmillan Learning Curriculum Solutions.

 


Guest Edited Series on Self-Assessment: Synthesis

by Ed Nuhfer, California State Universities (retired)

Self-assessment is a metacognitive skill that employs both cognitive competence and affective feelings. After over two decades of scholars’ misunderstanding, misrepresenting, and deprecating self-assessment’s value, recognizing self-assessment as valid, measurable, valuable, and connected to a variety of other beneficial behavioral and educational properties is finally happening. The opportune time for educating to strengthen that ability is now. We synthesize this series into four concepts to address when teaching self-assessment.

Image of a face silhouette watching a schematic of a man interfacing with mathematical symbols and a human brain
Image by Gerd Altmann from Pixabay

Teach the nature of self-assessment

Until recently, decades of peer-reviewed research popularized a misunderstanding of self-assessment as described by the Dunning-Kruger effect. The effect portrayed the natural human condition as most people overestimating their abilities, lacking the ability to recognize they do so, the most incompetent being the most egregious offenders, and only the most competent possessing the ability to self-assess themselves accurately.

From founding to the present, that promotion relied on mathematics that statisticians and mathematicians now recognize as specious. Behavioral scientists can no longer argue for “the effect” by invoking the unorthodox quantitative reasoning used to propose it. Any salvaging of “the effect” requires different mathematical arguments to support it.

Quantitative approaches confirm that a few percent of the populace are “unskilled and unaware of it,” as described by “the effect.” However, these same approaches affirm that most adults, even when untrained for self-assessment accuracy, are generally capable of recognizing their competence or lack thereof. Further, they overestimate and underestimate with about the same frequency.

Like the development of higher-order or “critical” thinking, the capacity for self-assessment accuracy develops slowly with practice, more slowly than required to learn specific content, and through more practice than a single course can provide. Proficiency in higher-order thinking and self-assessment accuracy seem best achieved through prolonged experiences in several courses.

During pre-college years, a deficit of relevant experiences produced by conditions of lesser privilege disadvantages many new college entrants relative to those raised in privilege. However, both the Dunning-Kruger studies and our own (https://books.aosis.co.za/index.php/ob/catalog/book/279 Chapter 6) confirm that self-assessment accuracy is indeed learnable. Those undeveloped in self-assessment accuracy can become much more proficient through mentoring and practice.

Teach the importance of self-assessment

As a nation that must act to address severe threats to well-being, such as healthcare, homelessness, and climate change, we have rarely been so incapacitated by polarization and bias. Two early entries on bias in this guest-edited series explained bias as a ubiquitous survival mechanism in which individuals relinquish self-assessment to engage in modern forms of tribalism that marginalize others in our workplaces, institutions, and societal cultures. Marginalizing others prevents holding the needed consensus-building conversations between diverse groups that bring creative solutions and needed action.

Relinquishing metacognitive self-assessment to engage in bias obscures perceiving the impacts and consequences of what one does. Developing the skill to exercise self-assessment and use evidence, even under peer pressure not to do so, seems a way to retain one’s perception and ability to act wisely.

Teach the consequences of devaluing self-assessment

The credibility “the effect” garnered as “peer-reviewed fact” helped rationalize the public’s tolerating bias and supporting hierarchies of privilege. A quick Google® search of the “Dunning Kruger effect” reveals widespread misuse to devalue and taunt diverse groups of people as ignorant, unskilled, and inept at recognizing their deficiency.

Underestimating and disrespecting other peoples’ abilities is not simply innumerate and dismal; it cripples learning. Subscribing to the misconception disposes the general populace to avoid trusting in themselves, in others who merit trust, and to dismiss implementing or even respecting effective practices developed by others presumed to be inferiors. It discourages reasoning from evidence and promotes unfounded deference to “authority.” Devaluing self-assessment encourages individuals to relinquish their autonomy to self-assess, which weakens their ability to resist being polarized by demagogues to embrace bias.

Teach self-assessment accuracy

As faculty, we have frequently heard the proclamation “Students can’t self-assess.” Sadly, we have yet to hear that statement confronted by, “So, what are we going to do about it?”

Opportunities exist to design learning experiences that develop self-assessment accuracy in every course and subject area. Knowledge surveys, assignments with required self-assessments, and post-evaluation tools like exam wrappers offer straightforward ways to design instruction to develop this accuracy.

Given the current emphasis on the active learning structures of groups and teams, teachers easily mistake these as the sole domains for active learning and deprecate study alone. The interactive engagements are generally superior to the conventional structure of lecture-based classes for cognitive mastery of content and skills. However, these structures seldom empower learners to develop affect or recognize the personal feelings of knowing that come with genuine understanding. Those feelings differ from those that rest on shallow knowledge and often launch the survival mechanism of bias at critically inopportune times.

Interactive engagement for developing cognitive expertise differs from the active engagement in self-assessment needed to empower individuals to direct their lifelong learning. When students employ quiet reflection time alone to practice self-assessment by enlisting understanding for content for engaging in knowing self, this too is active learning. Ability to distinguish the feeling of deep understanding requires repeated practices in such reflection. We contend that active learning design that attends to both cognition and affect is superior to design that attends only to one of these.

To us, John Draeger was particularly spot-on in his IwM entry, recognizing that instilling cognitive knowledge alone is insufficient as an approach for educating students or stakeholders within higher education institutions. Achievement of successful outcomes depends on educating for proficiency in both cognitive expertise and metacognition. In becoming proficient in controlling bias, “thinking about thinking” must include attention to affect to recognize the reactive feelings of dislike that often arise when confronting the unfamiliar. These reactive feelings are probably unhelpful to the further engagement required to achieve understanding.

The ideal educational environment seems one in which stakeholders experience the happiness that comes from valuing one another during their journey to increase content expertise while extending the knowing of self.


Knowledge Surveys Part 2 — Twenty Years of Learning Guiding More Creative Uses

by Ed Nuhfer, California State Universities (retired)
Karl Wirth, Macalester College
Christopher Cogan, Memorial University
McKensie Kay Phillips, University of Wyoming
Matthew Rowe, University of Oklahoma

Early adopters of knowledge surveys (KSs) recognized the dual benefits of the instrument to support and assess student learning produced by a course or program. Here, we focus on a third benefit: developing students’ metacognitive awareness through self-assessment accuracy.

Communicating self-assessed competence

Initially, we just authored test and quiz questions as the KS items. After the importance of the affective domain became more accepted, we began stressing affect’s role in learning and self-assessment by writing each knowledge survey item with an overt affective self-assessment root such as “I can…” or “I am able to…” followed by a cognitive content outcome challenge. When explaining the knowledge survey to students, we focus their attention on the importance of these affective roots for when they rate their self-assessed competence and write their own items later.

We retain the original three-item response scale expressing relative competence as no competence, partial competence, and high competence. Research reveals three-item scales as valid and reliable as longer ones, but our attraction to the shorter scale remains because it promotes addressing KS items well. Once participants comprehend the meaning of the three items and realize that the choices are identical for every item, they can focus on each item and rate their authentic feeling about meeting the cognitive challenge without distraction by more complex response choices.

photo of woman facing a black board with the words "trust yourself"
Image by Gerd Altmann from Pixabay

We find the most crucial illumination for a student’s self-assessment dilemma: “How do I know when I can rate that I can do this well?” is “When I know that I can teach how to meet this challenge to another person.”

Backward design

We favor backward design to construct topical sections within a knowledge survey by starting with the primary concept students must master when finally understanding that topic. Then, we work backward to build successive items that support that understanding by constantly considering, “What do students need to know to address the item above?” and filling in the detail needed. Sometimes we do this down to the definitions of terms needed to address the preceding items.

Such building of more detail and structure than we sensed might be necessary, especially for introductory level undergraduates, is not “handing out the test questions in advance.” Instead, this KS structure uses examples to show that deceptively disconnected observations and facts allow understanding of the unifying meaning of  “concept” through reaching to make connections. Conceptual thinking enables transferability and creativity when habits of mind develop that dare to attempt to make “outrageous connections.”

The feeling of knowing and awareness of metadisciplinary learning

Students learn that convergent challenges that demand right versus wrong answers feel different from divergent challenges that require reasonable versus unreasonable responses. Consider learning “What is the composition of pyrite?” and “Calculate the area of a triangle of 50 meters in length and a base of 10 meters?” Then, contrast the feeling required to learn, “What is a concept?” or “What is science?”

The “What is science?” query is especially poignant. Teaching specialty content in units of courses and the courses’ accompanying college textbooks essentially bypass teaching the significant metadisciplinary ways of knowing of science, humanities, social science, technology, arts, and numeracy. Instructors like Matt Rowe design courses to overcome the bypassing and strive to focus on this crucial conceptual understanding (see video section at times 25.01 – 29.05).

Knowledge surveys written to overtly provoke metadisciplinary awareness aid in designing and delivering such courses. For example, ten metadisciplinary KS items for a 300-item general geology KS appeared at its start, two of which follow.

  1. I can describe the basic methods of science (methods of repeated experimentation, historical science, and modeling) and provide one example each of its application in geological science.
  2. I can provide two examples of testable hypotheses statements, and one example of an untestable hypothesis.

Students learned that they would develop the understanding needed to address the ten throughout the course. The presence of the items in the KS ensured that the instructor did not forget to support that understanding. For ideas about varied metadisciplinary outcomes, examine this poster.

Illuminating temporal qualities

Because knowledge surveys establish baseline data and collect detailed information through an entire course or program, they are practical tools from which students and instructors can gain an understanding of qualities they seldom consider. Temporal qualities include magnitudes (How great?), rates (How quickly?), duration (How long?), order (What sequence?), frequency (How often?), and patterns (What kind?).

More specifically, knowledge surveys reveal magnitude (How great were changes in learning?), rates (How quickly we cover material relative to how well we learned it?), duration (How long was needed to gain an understanding of specific content?), order (What learning should precede other learning?), and patterns (Does all understanding come slowly and gradually or does some come in time as punctuated “Aha moments?”).

Knowledge survey patterns reveal how easily we underestimate the effort needed to do the teaching that makes significant learning change. A typical pattern from item-by-item arrays of pre-post knowledge surveys reveals a high correlation. Instructors may find it challenging to produce the changes where troughs of pre-course knowledge surveys revealing areas of lowest confidence become peak areas in post-course knowledge surveys showing high confidence. Success requires attention to frequency (repetition with take-home drills), duration (extending assignments addressing difficult contents with more time), order (giving attention to optimizing sequences of learning material), and likely switching to more active learning modalities, including students authoring their own drills, quizzes, and KS items.

Studies in progress by author McKensie Phillips showed that students were more confident with the material at the end of the semester rather than each individual unit. This observation even held for early units where researchers expected confidence would decrease given the time elapsed between the end of the unit and when the student took the post-semester KS. The results indicate that certain knowledge mastery is cumulative, and students are intertwining material from unit to unit and practicing metacognition by re-engaging with the KS to deepen understanding over time.

Student-authored knowledge surveys

Introducing students to the KS authoring must start with a class knowledge survey authored by the instructor so that they have an example and disclosure of the kinds of thinking utilized to construct a KS. Author Chris Cogan routinely tasks teams of 4-5 students to summarize the content at the end of the hour (or week) by writing their own survey items for the content. Typically, this requires about 10 minutes at the end of class. The instructor compiles the student drafts, looks for potential misconceptions, and posts the edited summary version back to the class.

Beginners’ student-authored items often tend to be brief, too vague to answer, or too focused on the lowest Bloom levels. However, feedback from the instructor each week has an impact, and students become more able to write helpful survey items and – more importantly – better acquire knowledge from the class sessions. The authoring of items begins to improve thinking, self-assessment, and justified confidence.

Recalibrating for self-assessment accuracy

Students with large miscalibrations in self-assessment accuracy should wonder, “What can I do about this?” The pre-exam knowledge survey data enables some sophisticated post-exam reflection through exam wrappers (Lovett, 2013). With the responses to their pre-exam knowledge survey and the graded exam in hand, students can do a “deep dive” into the two artifacts to understand what they can do.

Instructors can coach students to gain awareness of what their KS responses indicate about their mastery of the content. If large discrepancies between the responses to the knowledge survey and the graded exam exist, instructors query for some introspection on how these arose. Did students use their KS results to inform their actions (e.g., additional study) before the exam? Did different topics or sections of the exam produce different degrees of miscalibration? Were there discrepancies in self-assessed accuracy by Bloom levels?

Most importantly, after conducting the exam wrapper analysis, students with significant miscalibration errors should each articulate doing one thing differently to improve performance. Reminding students to revisit their post-exam analysis well before the next exam is helpful. IwM editor Lauren Scharff noted that her knowledge surveys and tests reveal that most psychology students gradually improved their self-assessment accuracy across the semester and more consistently used them as an ongoing learning tool rather than just a last-minute knowledge check.

Takeaways

We construct and use surveys differently than when we began two decades ago. For readers, we provide a downloadable example of a contemporary knowledge survey that covers this guest-edited blog series and an active Google® Forms online version.

We have learned that mentoring for metacognition can measurably increase students’ self-assessment accuracy as it supports growing their knowledge, skills, and capacity for higher-order thinking. Knowledge surveys offer a powerful tool for instructors who aim to direct students toward understanding the meaning of becoming educated, becoming learning experts, and understanding themselves through metacognitive self-assessment. There remains much to learn.

 


Metacognitive Discourse—Final Course Presentations that Foster Campus Conversations about Learning

by Gina Burkart, Ed.D., Learning Specialist, Clarke University

colored people and conversation bubblesPrior to the pandemic and now since returning to campus, there has been a shift in students’ use of group study and ability to learn and work in groups. When I began my position as Learning Specialist 10 years ago, it was not uncommon to find 30 students at group study sessions at 9 pm in the evening. Now, one group study session remains, and 2-3 students might attend the sessions (unless they are teams’ sessions required by athletic coaches). Colleagues have also shared in conversations that they have found it problematic that students avoid interacting with one another in the classroom and are not able to work and learn in physical groups. Further in my learning resource center year-end reports, data have shown a steady decline in group study attendance and a steady increase of students relying on support from me, the Learning Specialist. They want to work one/one with adults. In conversations with students and in online discussion blogs, students’ have shared a lack of inter- and intrapersonal communication skills as affecting their ability to work with their peers. In simple terms—overuse of electronic communication pre-pandemic and during the pandemic has left them unable to communicate interact with their classmates. This is problematic for a variety of reasons. In terms of learning, pedagogy is clear—learning is social (Bandura, 1977).

An Assignment to Reinforce Social Learning and Metacognition

In response, this semester, to reinforce social learning and metacognition, I changed the final assessment for the College Study Strategy course to be a final presentation that embedded metacognition and social discourse. The College Study Strategy course is metacognitive in nature in that it begins by having students reflect on their prior learning experiences, assess themselves and their skills, and set goals for the semester. It is a 1-credit course open to any student below 90 credits and can be retaken. However, in the second semester, it is almost entirely filled with students placed on academic probation or warning who are required to take the course. Curriculum includes theorists such as Marzano (2001), Bandura (1994), Ducksworth (2013), Dweck (2014), and Covey (2004) and requires them to begin applying new motivation, emotional intelligence, learning, reading, time management, study, note-taking, and test-taking strategies to their courses. In the past, students created a portfolio that demonstrated the use of their new strategies and presented their growth to me in a midterm and final conference. This year, I wanted them to share their new growth with more than me—I wanted them to share their growth with the entire community.

By changing the final project to be more outward-facing, the assignment would still be metacognitive in nature—requiring students to reflect on past learning, show how they made adjustments to learning and applied new methods and strategies, share in conversation how they made the adjustments, and finally explain how they will continue to apply strategies and continue their growth in the future with the new knowledge and strategies. Again,  it would require students to share with more than me. They would need to envision a larger audience and needs—the entire campus community (administrators, students, Athletic coaches, staff, professors, recruits) and create a presentation that could be adjusted to the audience. They would practice inter and intra-personal skills as they made adjustments to their presentation over the course of 2 hours while they remained at station in the library, prepared to share their presentation as members of the campus community approached. This also allowed for the campus community to benefit from the students’ new knowledge and growth of the semester. And, being on a small scale, it re-introduced students to the art of in-person, face-face conversation between each other and the value of seeking information from each other. This is something that has been eroding due to a heavy use of electronic communication and isolated learning that occurred during the pandemic.

Students were introduced to this assignment in week one of the semester. They were told that in week 6 they would choose any topic from the course curriculum that they felt they needed to focus on more intently based on their semester goals. Once choosing the curriculum they would focus on (ex: motivation, reading, procrastination, time management, studying, growth mindset), they would then research a different article each week related to their chosen topic (weeks 6-12) and apply the new critical reading strategy taught in class to create journal entries that would be used to prepare content for the final presentation. In weeks 14 or 15, they would present in the library at a table (poster session style) during a time of their choosing (two-hour block) to the campus community about their topic. The presentation needed to include some type of visual and the content needed to include all of the following metacognitive information about the topic:

  • past struggles
  • reasons for choosing the topic
  • strategies learned in class
  • information learned in their research
  • recommendations for other students struggling
  • strategies for continued growth

Positive Impact and Take-Aways

While students were nervous and hesitant prior to the presentations, during and after the presentations, they admitted to having fun sharing about their growth and learning. Staff, faculty, and students were also appreciative of the presentations and made a point of attending. Some future students/recruits even attended as they were touring. Not surprising, most students chose to present about motivation, time management and procrastination. A few students chose to present about growth mindset, Bloom’s Taxonomy as a study strategy, and reading. A surprising take-away was that in the metacognitive process of the presentation, many students connected improved reading strategies to increased motivation and reduction in procrastination.

While observing the presentations, it was encouraging to see students learn to adapt their presentations as people approached. Since they were stationed at a table for two hours, they needed to present the material many times to different types of audiences—and they had to field questions. As they presented and represented, they learned how to interact and present differently based on the needs of the audience. This adaptation required the use of metacognition and rhetorical analysis. It also built inter- and intrapersonal communication skills. It also came at a good time in the semester, as students were authentically seeking many of the strategies and skills to prepare for finals, conclude the semester, and look forward to the next semester. Many of the presenters had friends and team members, coaches, and faculty come to hear their presentations (as I had advertised the presentations to the campus in advance). In conclusion, metacognitive presentations that engage the entire campus community in discourse about learning may be a helpful step toward rebuilding learning communities post-pandemic. Next semester, I will continue this assignment. Additionally, next semester, I am working on embedding group reading labs into targeted courses to improve learning, motivation and reduce procrastination in the classroom.

 


Finding Your People

by Dr. Leah Poloskey, Assistant Clinical Professor, Department of Exercise and Rehabilitation Science, Merrimack College, and 

by Dr. Sarah Benes, Associate Clinical Professor, Department of Nutrition and Public Health, Merrimack College

(Post #1: Integrating Metacognition into Practice Across Campus, Guest Editor Series Edited by Dr. Sarah Benes)

How it all began . . .

Reflecting on the journey to having a Guest Editor spot with a mini-series of blog posts about metacognition with our colleagues from across campus was a great opportunity to reconnect to the power of community in this work. And it all began with a problem . . .

We had been discussing how challenging it was to engage students in our Health Science classes (Leah teaches in the Exercise and Rehabilitation Department and Sarah teaches in the Nutrition and Public Health Department). We decided to work together to investigate more deeply (rather than just dwelling on the challenge). We applied to host a Teaching Circle, which is an informal structure at Merrimack College that allows faculty and staff to come together around common interests. Teaching Circle facilitators are awarded small stipends for their time and effort in developing and running these opportunities. We believed that the Teaching Circle structure would provide a great opportunity for us to work within existing campus initiatives to enhance collaboration and engagement with faculty and staff across campus.schematic of three people facing each other with lightbulb being lit over their heads

Our first Teaching Circle was about student engagement. We ended up exploring mindset and the ways that mindset can impact engagement. We conducted a research study where we developed a tool that essentially is a measure of metacognitive states (Mandeville, et al., 2018). With this tool we learned how to assess a student’s self appraisal of their learning, which is a great opportunity to review a student’s intellectual development, mindset and metacognition. Now we had a way to assess these constructs, but what next?

We decided to apply for another Teaching Circle with a focus specifically on Metacognition. Our idea was approved and we were able to engage an even larger group of faculty, staff, and administrators from our academic support staff, to the psychology and business departments and more! Everyone in the group was interested in learning more about ways to support metacognition in our students in our various spaces. And this was the beginning of this blog post series!

What We Learned

Every meeting we had brought together a different group of people depending on schedules and availability. We had core folks who came each time and then a variety of others who came when they were able. Thinking about it now, we remember every meeting being exciting, dynamic, and invigorating.

We didn’t have set agendas and we didn’t have much reading or preparation (unless people asked for items to read). We really just came together to talk and share about our successes and challenges related to supporting students developing their metacognitive skills and to brainstorm ideas to try in our spaces. However, this opportunity for informal community gathering and building was a needed breath of fresh air. We always left energized for the work ahead (and we think the other participants did too!).

In fact, as a result of the Metacognition Teaching Circle, we embarked on a whole new project in which we used the MINDS survey (Mandeville, et al., 2018) at the beginning of the semester and then created “low touch” interventions to support metacognition and growth mindset depending on how students scored on the scale. From this we learned that many students are not familiar with concepts of metacognition and mindfulness, that many actually appreciated the tips and strategies we sent them (and some even used them!), and that students felt that more learning on these topics would be beneficial.

This then lead us to another study, this time examining faculty perceptions of metacognition which we were excited about because our experience suggested that it is likely that folks in certain settings or with certain backgrounds would be more familiar with metacognition and that faculty may not have the understanding or skills to teach metacognition in their courses. For faculty, it is so important to understand the idea of metacognition as it enables students to become flexible and self-directed learners. The teaching and the support of metacognition in the classroom is impactful. It allows students to become aware of their own thinking and to become proficient in choosing appropriate thinking strategies for different learning tasks. Unfortunately, this line of inquiry did not last long due to COVID 19 but we hope to pick this back up this year as we feel it is an important area that could be impactful for faculty and students.

While the research ideas and changes to practice are exciting and were impactful benefits of our Teaching Circles, one of our biggest takeaways was the reminder of the importance of finding others who are also doing the work. Sometimes on our campus, and we suspect it is the case at other institutions as well, we get siloed and often our meetings are with the same folks about the same topics. Being able to facilitate and participate in a cross-campus initiative about a passion topic was an amazing opportunity to meet new people, make new connections, gain different perspectives and create new ideas and strategies to try. We found many people doing great work with students on our campus across so many different departments and schools, and most importantly, found “our people” – people who you can go to when you are stuck, people who you can bounce ideas off of and collaborate with . . . we found our “metacognition people” (some of them at least).

While this was not a “new” idea or “cutting edge”, coming off a year in which we have been separated (in so many ways), we were reminded of the power of connections with others to maintain and sustain ourselves as academics and as humans. We wanted to share that in the guest series by not only showcasing some of the work that our colleagues are doing but also to remind readers to try and find your people . . . whether they are on your campus or off, whether you meet in person or virtual – or only via Tweets on Twitter . . . find the people who can help you maintain, sustain and grow your interest, skills, passion and joy!

We hope you enjoy reading the work of our colleagues and that it helps you on your journey.

References

​​Mandeville, D., Perks. L., Benes, S. & Poloskey, L. (2018). The Mindset and Intellectual Development Scale (MINDS): Metacognitive Assessment for Undergraduate Students. International Journal of Teaching and Learning in Higher Education, 30(3).


Why Metacognition?

By: Melissa Terlecki, PhD, Cabrini University PA

“Learning about myself wasn’t easy. Metacognition took way more work than all my other classes, but I learned so much about myself that I hope to apply in the future” – Anonymous Student Quote.

The Question at Hand

What do we want our students to get out of college? Does it extend beyond content – to include skills to potentially last a lifetime? I believe so, and argue that self-awareness and metacognition development should be part of what every college student achieves.

The words "Who Me?" on a yellow backgroundSelf-awareness involves understanding one’s strengths and areas for improvement; it’s recognizing how we grow best and optimizing our potential. Metacognition is more than just “thinking about thinking” – it’s applying that self-knowledge to better oneself. Skills and strategies related to self-awareness and metacognition may not come naturally – or easily. Explicitly promoting them both through coursework in college may be a start.

Below are short overviews of two initiatives I have led at my institution, along with some student and faculty feedback and a brief personal reflections. I encourage you to think about ways to incorporate self-awareness and metacognition.

Metacognition in Leadership

I embarked on a journey to include metacognition as part of a Leadership program based on the Social Change Model of Leadership Development. Self-awareness is core to living and leading up to one’s greatest potential in strengths-based development. Metacognition, based on this model, is focused on building positive change in society as a leader at any level. Thus, I built a course around developing self-awareness while linking recognition of one’s skills to leadership potential. My “Metacognition in Leadership” course is open to any of our students, and although part of a Leadership minor, many seek to take it, despite the rumors of the workload.

Metacognition is built in every activity and assignment and simulates a flipped classroom. Course feedback shows that students are challenged yet do really well in the course, including self-measured improvements in metacognition. Grades are high and students argue that metacognition should be taught to everybody.

“Why is this not taught in grade-school? I would’ve done much better…” – Michelle Brzoska, Student.

“I believe for me it was challenging to dig deep into my traits and values. We just do things and say things without thinking about them. However, this course pushed me to consider those traits and reflect on them. Most of the time we do not have the time to reflect on ourselves and I believe sometimes it leads to false perceptions about ourselves.” – Maria Khan, Student.

Metacognition in First-Year Experience

I also sought to embed self-awareness in first-year experience (FYE) programming, as we know self-awareness of interests and strengths can lead to persistence (and retention) in academic settings. In 2020, our FYE programming was about to undergo revision. I was asked to step in to provide a metacognitive framework for students’ college success, as realizing their best path to learning and to their eventual major/s is the goal of such a course.

Students learned about topics such as self-regulation, emotional intelligence, motivation and achievement, among other areas, which were directly connected to student self-assessments. Faculty utilized regular reflections and feedback, with weekly check-ins and academic advising. Course feedback from both instructors and students was favorable: students enjoyed learning about themselves and faculty appreciated this, yet commented on the challenging nature of teaching metacognition (given faculty teaching our FYE course are from all different disciplines):

“Content was good but challenging to take on. Students got a lot out of it in only one semester. Faculty need more training in metacognition” – Anonymous Faculty Quote.

Reflections on Interventions

Adding metacognitive content AND pedagogy takes work. And time. And a lot of grading and feedback. It is an iterative process and is not static or traditional by any means. Students may resist active engagement that forces them beyond their comfort zones in passive learning, and especially self-awakening. This is a different type of learning than students are used to or expect at the college level, perhaps given their previous experiences at K-12. It takes more effort on both the student’s and the instructor’s parts.

Both students and faculty need training in metacognition: the pedagogy, the routine self-reflection and feedback. I provide metacognition training/workshops to institutions for both faculty and staff. I have seen and heard the impact these techniques can have on teaching and learning. For metacognition to stick, however, it needs to be more than quick tricks – metacognition is a “lifestyle” change in pedagogy for teachers. It has to be a new way of learning and self-discovery for learners. Again, this is not easy, but is well worth the effort, as benefits of self-awareness extend beyond the classroom to our relationships, our jobs, and our lives.

“I think it’s important for people to learn metacognition because of the awareness and understanding you get from it. Once you learn more about it, it’ll be easier to control your emotions and be able to comprehend how others feel” – Orlyany Sanchez, Student.

“I hope I can keep reflecting after this course is over. It has already changed how I think and feel about other people” – Anonymous Student Quote.

“It is extremely vital that we learn about metacognition to understand ourselves better so we can interact with others well. There were times where I was not able to connect with others because I was not able to connect with myself. But this [course] showed me the importance of knowing oneself as well as applying to my future” – Maria Khan, Student.

—————–

Note: Feel free to contact the author, Melissa Terlecki, for more information on course materials and/or metacognition workshop availability to bring to your school!  (mst723@cabrini.edu)

Note 2: Catch my talk on metacognition at the 2021 American Psychological Association virtual conference as the Harry Kirke Wolfe Lecturer! (12-14 August; see https://convention.apa.org/) 


Promoting Learning Integrity Through Metacognition and Self-Assessment

by Lauren Scharff, Ph.D., U. S. Air Force Academy*

When we think of integrity within the educational realm, we typically think about “academic integrity” and instances of cheating and plagiarism. While there is plenty of reason for concern, I believe that in many cases these instances are an unfortunate end result of more foundational “learning integrity” issues rather than deep character flaws representing lack of moral principles and virtues.

photo of a hand holding a compass with a mountain scene background (by Devon Luongo)Learning integrity occurs when choices for learning behaviors match a learner’s goals and self-beliefs. Integrity in this sense is more like a state of wholeness or integrated completeness. It’s hard to imagine this form of integrity without self-assessment; one needs to self-assess in order to know oneself. For example, are one’s actions aligned with one’s beliefs? Are one’s motivations aligned with one’s goals? Metacognition is a process by which we gain awareness (self-assess) and use that awareness to self-regulate. Thus, through metacognition, we can more successfully align our personal goals and behaviors, enhancing our integrity.

Metacognitive Learning and Typical Challenges

When students are being metacognitive about their learning, they take the time to think about (bring into awareness) what an assignment or task will require for success. They then make a plan for action based on their understanding of that assignment as well their understanding of their abilities and current context. After that, they begin to carry out that plan (self-regulation). As they do so, they take pauses to reflect on whether or not their plan is working (self-awareness/self-assessment). Based on that interim assessment, they potentially shift their plan or learning strategies in order to better support their success at the task at hand (further self-regulation).

That explanation of a metacognitive learning may sound easy, but if that were the case, we should see it happening more consistently. As a quick example, imagine a student is reading a text and then realizes that they are several pages into the assignment and they don’t remember much of what they’ve read (awareness). If they are being metacognitive, they should come up with a different strategy to help them better engage with the text and then use that alternate strategy (self-regulation). Instead, many students simply keep reading as they had been (just to get the assignment finished), essentially wasting their time and short-cutting their long-term goals.

Why don’t most students engage in metacognition? There are several meaningful barriers to doing so:

  • Pausing to self-assess is not a habitual behavior for them
  • It takes time to pause and reflect in order to build awareness
  • They may not be aware of effective alternate strategies
  • They may avoid alternate strategies because they perceive them to take more time or effort
  • They are focused on “finishing” a task rather than learning from it
  • They don’t realize that some short-term reinforcements don’t really align with their long-term goals

These barriers prevent many students engaging in metacognition, which then makes it more likely that their learning choices are 1) not guided by awareness of their learning state and 2) not aligned with their learning goals and/or the learning expectations of the instructor. This misalignment can then lead to a breakdown of learning integrity with respect to the notion of “completeness” or “wholeness.”

For example, students often claim that they want to develop expertise in their major in order to support their success in their future careers. They want to be “good students.” But they take short-cuts with their learning, such as cramming or relying on example problem workout steps, both of which lead to illusions of learning rather than deep learning and long-term retention. These actions are often rewarded in the short term by good grades on exams and homework assignments. Unfortunately, if they engage in short-cutting their learning consistently enough, when long-term learning is expected or assessed, some students might end up feeling desperate and engage in blatant cheating.

Promoting Learning Integrity by Providing Support for Self-Assessment and Metacognition

Promoting learning integrity will involve more than simply encouraging students to pause, self-reflect, and practice self-regulation, i.e. engage in metacognition. As alluded to by the list of barriers above, being metacognitive requires effort, which also implies that learning integrity requires effort. Like many other self-improvement behaviors, developing metacognition requires multiple opportunities to practice and develop into a way of doing things.

Fortunately, as instructors we can help provide regular opportunities for reflection and self-assessment, and we can share possible alternative learning strategies. Together these should promote metacognition, leading to alignment of goals and behaviors and to increased learning integrity. The Improve with Metacognition website offers many suggestions and examples used by instructors across the disciplines and educational levels.

To wrap up this post, I highlight knowledge surveys as one way by which to promote the practice and skill of self-assessment within our courses. Knowledge surveys are shared with students at the start of a unit so students can use them to guide their learning and self-assess prior to the summative assessment. Well-designed knowledge survey questions articulate granular learning expectations and are in clear alignment with course assessments. (Thus, their implementation also supports teaching integrity!)

When answering the questions, students rate themselves on their ability to answer the question (similar to a confidence rating) as opposed to fully writing out the answer to the question. Comparisons can be made between the confidence ratings and actual performance on an exam or other assessment (self-assessment accuracy). For a more detailed example of the incorporation of knowledge surveys into a course, as well as student and instructor reflections, see “Supporting Student Self-Assessment with Knowledge Surveys” (Scharff, 2018).

By making the knowledge surveys a meaningful part of the course (some points assigned, regular discussion of the questions, and sharing of students’ self-assessment accuracy), instructors support the development of self-assessment habits, which then provide a foundation to metacognition, and in turn, learning integrity.

———————————————–

* Disclaimer: The views expressed in this document are those of the author and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


How Metacognition Can Foster Inclusivity in the Classroom

by Christopher Lee; Snow College

Kelly Field (2018) reports that “A growing body of research suggests that students who feel they belong at their college are more likely to remain there [and] that first-generation and minority students are less likely to feel a connection to their colleges” (para. 27). As an instructor at a 2-year college, I recognize the important role that my institution plays in functioning as a bridge to further educational opportunities, particularly for underrepresented students. Crucial to this mission is ensuring that I do my part to facilitate a classroom environment in which these students feel valued and included.

Inclusivity means working to ensure that curricula and teaching practices don’t exclude marginalized minority students and help to close existing achievement gaps. It means not only valuing diversity but creating a space for diverse groups of students to actually feel included. It entails serious introspection from faculty (before we even enter the classroom) about implicit biases we may hold toward others, opportunities for privileged students to examine their attitudes about underprivileged peers, and opportunities for minority students to critically reflect on their own academic abilities. An inclusive classroom, then, is contingent on honest metacognitive reflection from both faculty and students.

a hand holding a mirror

Faculty: Holding Up the Mirror

Inclusivity requires holding the mirror up to ourselves as instructors and asking how our behaviors, teaching practices, and curriculum choices may confirm or exacerbate student feelings of exclusion. As we strive for an inclusive classroom – in relation to race, class, gender identity, sexual orientation, religious affiliation, age, culture, or ideology – it’s critical that we examine the hidden biases we may hold about certain groups of students and recognize how these biases manifest in the classroom.

It’s one thing to acknowledge that we may have negative biases, but can we actually identify and control them? Patricia Divine’s (cited in Nordell, 2017) research suggests that it is possible to identify and mitigate biases, noting that they can be overridden, but not overwritten. In other words, completely removing our biases doesn’t seem to be a realistic goal, but we can moderate them, once recognized. Divine offers a model for faculty that incorporates key components of metacognitive thinking.

First, we must become aware of our own implicit biases. Although there’s no silver bullet, the Implicit Association Test at Harvard’s Project Implicit can be a useful resource. Second, we must become concerned about the implications and outcomes of our biases, acknowledging that there are very real and harmful consequences to holding unchecked biases. Finally, we must work to replace biases with more productive attitudes that align with our conscious or aspirational values. Subsequently, we can design strategies to monitor and assess our progress.

Metacognitive Practices for Students

The work of creating an inclusive, “decolonized” classroom (Seward, 2019) can’t be reduced to a short and simple list; however, these three practical suggestions can be effectively implemented in any course in an effort to utilize the benefits of metacognition toward increasing inclusivity.

  • Assign Reflective Exercises: Start students reflecting on their thinking processes and assumptions early in the semester, particularly in relation to their abilities and potentially flawed preconceptions about themselves, others, and college. I have students write a short essay about their writing and thinking processes, previous experiences with English courses, including negative internalized experiences, and their expectations about our current class. Students can choose to share their thoughts and experiences openly with each other, demystifying the idea that there’s one “correct” (i.e. white, male, middle class, etc.) way to approach writing, thinking, and other academic skills. Previous negative experiences aren’t necessarily exclusive to them individually and won’t act as permanent barriers to their educational goals.

With opportunities to metacognitively reflect, students are more likely to feel included in the classroom environment, early on, if they see a variety of effective approaches to learning tasks. With this understanding, they need not feel pressure to conform to the norms of a hidden curriculum (Margolis, 2001).

  • Invite Former Underrepresented Students to Speak: Former students, particularly those who are underrepresented, can be a powerful reference point and model for current students, in both bolstering the self-efficacy of underrepresented students and busting negative minority stereotypes held among other students. Encourage students who have successfully navigated your course to candidly discuss successes, failures, and effective learning strategies. This could be followed-up with a quick one-minute reflection paper that students complete in which they acknowledge their own struggles and make plans for addressing them.

Although we need to be careful not to inappropriately spotlight students (which usually results from us “volunteering” students), this can help underrepresented students to feel more represented and included. We can also use underrepresented student work as models, particularly work that reinforces the idea that there can be multiple ways to reach course goals.

  • Engage Students in High Impact Practices: Design projects that allow for greater engagement. Opportunities to participate in undergraduate research, for instance, require students to design, monitor, and adjust their work with faculty mentoring and peer feedback. I incorporate such research opportunities in my freshmen research writing courses to various degrees. As Draeger (2018) notes, “undergraduate research allows students the opportunity to become co-inquirers within an existing scholarly conversation” (para. 4). Actively contributing to an existing academic conversation, rather than passively reporting, requires a number of metacognitive traits, such as identifying and working to mitigate existing biases about topics, assessing what they already know or think they know, how to weigh and prioritize information (including where research gaps exist in the broader conversation), and how to adjust a research question when source material presents new and often contradictory evidence. I scaffold assignments with reflective components to serve as individual checkpoints along this path.

First generation and other minority students, in particular, have been shown to benefit from undergraduate research because of increased interactions with faculty and the institution, developing closer relationships with peers, and the opportunity to challenge existing knowledge and power structures with their own primary research contributions (Charity Hudley et al., 2017). These outcomes help to alleviate some of the most prominent barriers to an inclusive classroom.

Practices like these, in addition to reflecting on our own potentially excluding attitudes and behaviors, can aid us in shaping our classroom spaces to be more inclusive and, ideally, help further serve the mission of colleges and universities in recruiting, retaining, and advancing minority students.

References

About Us. (2011). Project Implicit. Retrieved July 7, 2020, from https://implicit.harvard.edu/implicit/aboutus.html

Draeger, J. (2018, June 22). Metacognition supports HIP undergraduate research. Improve with Metacognition. https://www.improvewithmetacognition.com/metacognition-supports-hip-undergraduate-research/

Field, K. (2018, June 3). A third of your freshmen disappear. How can you keep them? The Chronicle of Higher Education. https://www.chronicle.com/article/A-Third-of-Your-Freshmen/243560

Charity Hudley, A.H., Dickter, C.L., & Franz, H.A. (2017). The indispensable guide to undergraduate research: Success in and beyond college. New York: Teachers College Press.

Margolis, E. (2001). The hidden curriculum in higher education. New York: Routledge.

Nordell, J. (2017, May 7). Is this how discrimination ends? The Atlantic. https://www.theatlantic.com/science/archive/2017/05/unconscious-bias-training/525405/

Seward, M. (2019, April 11). Decolonizing the Classroom: Step 1. National Council of Teachers of English. https://ncte.org/blog/2019/04/decolonizing-the-classroom/


Series Introduction – Ways Metacognition Can Enhance Student Success

by Anton O. Tolman, Ph.D., Utah Valley University Guest Editor

There appears to be growing interest among faculty and researchers on the topic of metacognition. This is evidenced, in part, by increasing research and published works related to the subject such as Saundra and Stephanie McGuire’s (2015) book regarding teaching students how to learn. Other recent works (e.g., Doyle & Zakrajsek, 2018; Bain, 2012) are aimed primarily at students, encouraging them to recognize how the brain works and how they can adopt behaviors and strategies that will enhance their learning. These are laudable efforts that provide a solid foundation for faculty to introduce to students thereby increasing students’ chances of success. Yet, faculty and others who approach metacognition only from the perspective of enhancing student learning strategies or behaviors (process metacognition) are missing the opportunity for a deeper understanding of metacognition’s central role in learning.

With a broader understanding, all faculty and staff who have contact with students can promote and advocate for metacognitive skill development in general education, course development, across programs and curricula, and as valued skills in students’ personal and professional lives.

Three Ways Metacognition Can Enhance Student Success

Here are three quick examples of how metacognition furthers student success as well as promoting the overarching goals of colleges and universities:

  1. Fostering process metacognition helps students understand how they learn and promotes the acquisition and development of effective learning strategies across subjects (including General Education) as well as within the major. This promotes content mastery and improved academic skills and performance as well as transfer across knowledge domains, but only if the use of these skills is perceived as valued by instructors across courses and within the major. Otherwise, students tend to see this emphasis as restricted to a particular course or professor. If student advisors also encouraged buy-in of the value of these skills and their value to professional careers, this could also have a significant impact.
  2. Metacognition reduces student resistance to learning. Students, especially in their first years, often see themselves as consumers, functioning primarily in a passive “student” role they know well and are comfortable with. Resistance to learning is ubiquitous in education and plays a major role in decreasing student motivation to learn. Resistance arises due to systemic influences (see Tolman & Kremling, 2017), one of which is the lack of metacognitive awareness (see Figure below).

flow chart showing components of the Integrated_Model_of_Student_Resistance

Students’ lack of self-awareness of learning strategies, their relative effectiveness, and the ability to monitor and evaluate their learning (beyond grades) naturally leads to negative classroom situations, frustration, and anxiety. In their consumer or “student” role, pushed in part by social expectations and institutional culture, many believe that if they have put in good effort, they should receive excellent grades. If this does not occur, natural targets of that frustration are the instructor (she doesn’t teach well), the content (I’m no good at this subject), or the generalization that they do not belong in college.

Promoting student metacognition, especially, shifts the responsibility for learning back towards the student who hopefully realizes they can succeed by using better learning approaches and encourages them to seek help when they realize they have not mastered important skills or concepts. This also increases student motivation and desire to learn and can curtail the sense that they do not belong. Instructors, advisors, and others who emphasize the relevance of metacognitive skills in professional careers, or even effective parenting, can help students see value and meaning in using these skills in many environments and across their lives.

  1. Another vital aspect of metacognition is that in becoming self-aware of their own motives, approaches, level of resistance, and personal responsibility, students begin to shift their personal narrative and identity away from that of consumer to that of someone capable of success. They begin to see themselves as someone who can be a lifelong learner and a learned person in their profession and in society. Taraban (2020; Taraban & Blanton 2008) described this process of personal narrative development as inherently metacognitive. In addition, Hale (2012) likewise explores the powerful interdependent relationships between metacognition, critical thinking, and personal narrative.

These relationships are so interdependent and so potent, they underlie the documented effectiveness of what are called “high impact practices” in learning and retention. A good example of this is the power of undergraduate research, an enterprise heavily laden with metacognitive experiences if done well, to shape students’ personal narratives and create a new sense of identity as a scholar, as someone capable of asking their own questions and finding answers. These experiences are especially powerful for first-generation and minority students as clearly described by Charity Hudley, Dickter, and Franz (2017) and the work of Tarabon and Blanton (2008).

Overview of this Guest Editor Series

Even with these limited examples, it should become obvious that metacognition is central to successful learning. The purpose or goal of this Mini-series is to explore several pivotal aspects of learning in higher education related to student resistance and motivation and to encourage all faculty and students to explore these boundaries. In the upcoming blogs, you will hear from the following authors on several important subjects:

  • Christopher Lee on How Metacognition Can Facilitate Student Inclusion in the Classroom
  • Steven Pearlman on Metacognition and the Fish in the Water
  • Benjamin Johnson on Change Instead of Continuity: Using Metacognition to Enhance Student Motivation for Learning
  • Anton Tolman on Boosting the Effectiveness of Collaborative Learning Using Metacognition

I will conclude the series with a blog focusing primarily on personal narrative and self-identity. Above, I noted that student resistance is a common occurrence in our classrooms. However, resistance is not limited to students. It is time that we, as professors, go beyond the constraints of thinking of ourselves as “content experts” and consider the broader scope of what we are capable of achieving by promoting metacognition in our assignments, our curriculum, across the major, and our institutions. We hope this blog series will help you see some new possibilities.

References

Bain, K. (2012). What the Best College Students Do. Belknap Press.

Charity Hudley, A.H., Dickter, C.L., & Franz, H.A. (2017). The Indispensable Guide to Undergraduate Research: Success In and Beyond College. New York: Teachers College Press.

Doyle, T. & Zakrajsek, T.D. (2018). The New Science of Learning: How to Learn in Harmony with Your Brain (2nd Ed). Sterling, VA: Stylus Publishing.

McGuire, S. Y. & McGuire, S. (2015). Teach Students How to Learn: Strategies You Can Incorporate Into Any Course to Improve Student Metacognition, Study Skills, and Motivation. Sterling, VA: Stylus Publishing.

Taraban, R., & Blanton, R. L. (Eds.). (2008). Creating Effective Undergraduate Research Programs in Science: The Transformation from Student to Scientist. New York: Teachers College Press.

Taraban, R. (2020, June 25). Metacognition and the Development of Self. ImproveWithMetacognition.com. https://www.improvewithmetacognition.com/metacognition-and-self-identity/

Tolman, A.O. & Kremling, J. (2017). Why Students Resist Learning: A Practical Model for Understanding and Helping Students. Sterling, VA: Stylus Publishing.


Want to Kick Start a Conversation about Metacognition? Assign the Blind Draft

by Amy Ratto Parks, Ph.D., University of Montana

Although many of us feel held in suspense about the state of teaching and learning in the fall, there is one thing I know for sure: I will still be using the Blind Draft assignment. The Blind Draft is a homework assignment that requires students to compose a short draft into a completely blackened computer screen; without any visual cue from the writing, they quickly become aware of their thinking. This single assignment builds classroom community, grounds students in their own minds and bodies, introduces them to a new way of understanding themselves as learners, and kicks off a course-long conversation about metacognition.

Photo of a woman with a cloth blindfold around her eyes

Right now, teaching and learning are happening in a remarkably distracting set of circumstances. National and global issues lead us into internet news and social media cultures that have had wide-ranging and varied impacts on teachers and students in Western cultures. Through the visually dominant world inside our screens we are propelled forward (Brockman, 2011) away from our own minds and bodies and into carefully sculpted Instagram lives, snappy Twitter feeds and sharp info graphics. But in order for teaching and learning to happen well, our minds have to move away from the glossy surface and focus on the task at hand. We know that students who are able to monitor their attention by developing skills in attention literacy (Brockman, 2011) also demonstrate strong meta-cognitive awareness and are positioned for a productive learning experience (Tarricone, 2011).

The Blind Draft offers students an initial small step into meeting their own minds as learners. It creates a unique and memorable composing and revision experience that provides a platform for experience-based reflection and conversation about how differently individual student’s minds might respond to novel learning challenges (Yancey, 1998). Those conversations provide an opening into a discussion about how metacognitive concepts help us understand how to navigate writer’s block, writing anxiety, and other common writing barriers. Supporting a sustained conversation about how students respond to writing challenges will help athleticize their attention (Caldwell, 2018), build personal understanding, and help them develop an increased sense of agency (Negretti, 2012) over their own identities and abilities as writers.

The Assignment

One of the most beautiful things about the Blind Draft is the simplicity of its design, however, it does work best with a bit of set up at the start because students immediately question the simplicity.

The set up

Because students focus so carefully on their grades, there is an underlying sense of risk in any work assigned in a class. Therefore, it is important to frame the assignment as an experiment where the goal is to participate in the experiment so that you all will have something interesting to talk about during the next class. You will also want to think of a very (very) general writing prompt such as, write about “you and food” or “you and happiness” for students to respond to in their drafts.

The directions

In general, the work happens in three parts: 1) Students type in response to the prompt without being able to see what they’re typing for 15 minutes. 2) They revise the messy blind draft. 3) They reflect briefly on their own writing behavior You might offer them the following directions:

Step 1: Compose (Note: The blind drafting process works best when typing on a computer)

  1. Set a timer for 15 minutes.
  2. Open and save a blank document. (Note: Saving the document before typing is important because sometimes typing blindly leads one to delete the entire document!)
  3. Make a mental note of the writing prompt.
  4. Turn off or cover the screen. 
  5. Type without stopping for 15 minutes. 
  6. Print or save document as “blind draft.” (Note: This will be your preference depending on whether you are collecting hard copy or electronic versions. The important thing is simply that they specify which draft was the blind draft and which was the revision. It seems as though that would be obvious, but sometimes it isn’t.)

Step 2: Revise

  1. Revise the draft into 1-2 page essay. (You can keep everything from the first draft or nothing at all and rearrange as you please.)
  2. Print or save document as “revised blind draft.”

Step 3: Reflect

  1. List the kinds of changes you made between drafts.
  2. Did you like this exercise? Hate it? Something in between? 
  3. Where did you rest your eyes without a screen to study?  
  4. Print or save document as “reflection.”

These instructions often cause laughter or anxiety — or both. Students need reassurance that yes, that first draft will be a terrific and spectacular mess, and that yes, that is the goal; they also appreciate a reminder that the second draft allows them to share a stronger more controlled version of their writing.

Student Responses

It turns out that students either love or hate this assignment; there is very little in between. I begin the discussion of the drafts by asking people to raise their hands if they hated this assignment, then if they loved it, and this initial question is often enough to begin a robust conversation about how differently they engaged with the writing. Why would some people feel constrained by this and other feel liberated? Right from the start, they are plenty baffled by anyone who had an experience that differed from their own.

Specifically, students say similar things about the assignment. Mostly, they notice everything their minds were doing besides writing. The often found themselves asking:

  • How long is 15 minutes? They report that they fixate on wondering how much time has passed and whether or not they are writing quickly. (In other words, am I doing a good job?)
  • How much am I writing? They report fixating on how much they have written, even when there is no assigned length requirement. (In other words, am I doing a good job?) They also realize that while they were distracted by the blind writing, they didn’t notice how much they were writing. Some students report writing less than they imagined, but most write more (and often report that they had no idea how often they’re thinking about page length rather than the topic of the writing).
  • Why did I try to fix the errors? In every class, students will laughingly recount trying to backspace and delete an error – even when they are not sure they made an error. They describe trying to count backward the correct number of letters to fix a spelling mistake even when they realize that it is unlikely that they will have actually corrected the error. (In other words, am I doing a good job?)

Take-away Messages

In a mini-lecture afterward, I explain that:

  • The human cognitive processes are messy and unpredictable (Flavell, 1976). Therefore, we need to expect a certain amount of confusion or chaos in the learning process. Reinforcing this early in a class helps students normalize challenge and difficulty.
  • The cognitive processes that inform writing of any kind are also are messy and unpredictable (Flower & Hayes, 1981). Therefore, we also need to leave room for some confusion and chaos in the writing process! This idea normalizes the fact that writing is always challenging and pushes back against the myth that writing is just easy for some people.
  • The process of writing is not linear – it’s recursive. We don’t just write; we write, re-read, write, re-read (Olive, 2014). Therefore, though we all want to sit down and “hammer out” an essay, strong writing doesn’t happen that way.
  • Our eyes don’t stay on the words we’re typing; they are skipping back and ahead (de Smet, M. J. R., Leijten, M., & Van Waes, L. (2018). Therefore, if you’re on a first draft and haven’t written much, your eyes keep looking back and ahead at nothing which can induce panic and mental paralysis.
  • It is important to sometimes “write badly” (Ballenger, 2018). Leaving space for chaos in drafting allows us to become aware of our own mental processes, thoughts and ideas – and can make space for new connections and ideas to come forward.

You might wonder what I do after collecting this assignment since it produces a lot of material at once. Since the goal was the experience of the work, I do not line edit their writing or respond to each draft; instead, I thank them for investing in the experiment and I reinforce how beautifully messy their blind drafts were. The students really do take quite a risk by just turning in something as messy as a blind draft and it is nice to remind them that by turning it in that have confronted the chaos of their own writing minds, and therefore, have already done a good job.

Perhaps more importantly, the Blind Draft assignment helps them develop a sustained awareness of how their minds are working while writing and for many students that extends into their ability to monitor their thinking and behavior while completing other academic tasks (i.e. metacognition).

Works Cited

Ballenger, B. P. (2018). The curious writer. Boston: Pearson.

(2011) Is the internet changing the way we think? The net’s impact on our minds and future. Brockman,   John (Ed.) New York, NY. Harper Perennial.

Caldwell, C. (2018). Bodyfulness. Boulder, CO: Shambhala.

de Smet, M. J. R., Leijten, M., & Van Waes, L. (2018). Exploring the Process of Reading During Writing Using Eye Tracking and Keystroke Logging. Written Communication, 35(4), 411–447. https://doi.org/10.1177/0741088318788070

Olive, T. (2014). Toward a parallel and cascading model of the writing system: A review of research on writing processes coordination. Journal of Writing Research, 6(2), 173-194. doi:10.17239/jowr-2014.06.02.4

Negretti, R. (2012). Metacognition in student academic writing: A longitudinal study of  Metacognitive awareness and its relation to task perception, self-regulation, and evaluation of performance. Written Communication, 29(2), 142–179.

Tarricone, P. (2011). The taxonomy of metacognition. New York: Psychology Press.

Yancey, K.B. (1998). Reflection in the writing classroom. Logan: Utah State  University Press.


How to Use Post-Test Analysis to Facilitate Metacognition in the College Classroom

by Gina Burkart, Ed.D., Learning Specialist, Clarke University

Pedagogy for Embedding Strategies into Classes

The transition to college is difficult. Students quickly discover that their old strategies from high school do not serve them well in college when they fail their first exam. As the Learning Specialist, I guide these students in modifying strategies and behaviors and in finding new strategies. This also involves helping them move away from a fixed mindset where they believe some students are just born smarter than others and move toward a growth mindset where they reflect on habits and strategies and how to set goals and make changes to achieve desired outcomes. Reflective metacognitive discussion and exercises that develop a growth mindset are necessary for this type of triaging with students (Dweck, 2006; Masters, 2013; Efklides, 2008; VanZile-Tamsen & Livingston, 1999; Livingston, 2003).

As the Learning Specialist at the University, I work with students who are struggling, and I also work with professors in developing better teaching strategies to reach students. When learning is breaking down, I have found that oftentimes the most efficient and effective method of helping students find better strategies is to collaborate with the professor and facilitate strategy workshops in the classroom tailored to the course curriculum. This allows me to work with several students in a short amount of time—while also supporting the professor by demonstrating teaching strategies he or she might integrate into future classes.

magnifying glass with the words Exam Analysis shown

An example of a workshop that works well when learning is breaking down in the classroom is the the post-test analysis workshop. The post-test analysis workshop (see activity details below) often works well in classes after the first exam. Since most students are stressed about their test results, the metacognitive workshop de-escalates anxiety by guiding students in strategic reflection of the exam. The reflection demonstrates how to analyze the results of the exam so that they can form new habits and behaviors in attempt to learn and perform better on the next exam. The corrected exam is an effective tool for fostering metacognition because it shows the students where errors have occurred in their cognitive processing (Efklides, 2008). The activity also increases self-awareness, imperative to metacognition, as it helps students connect past actions with future goals (Vogeley, Jurthen, Falkai, & Maier, 1999). This is an important step in helping students take control of their own learning and increasing motivation (Linvingston & VanZile Tamsen, 1999; Palmer & Goetz, 1988; Pintrich & DeGroot, 1990).

Post-Test Analysis Activity

When facilitating this activity, I begin by having the professor hand back the exams. I then take the students through a serious of prompts that engage them in metacognitive analysis of their performance on the exams. Since metacognitive experiences also require an awareness of feeling (Efklides, 2008), it works well to have students begin by recalling how they felt after the exam:

  • How did you feel?
  • How did you think you did?
  • Were your feelings and predictions accurate?

The post-test analysis then prompts the students to connect their feelings with how they prepared for the exam:

  • What strategies did you use to study?
    • Bloom’s Taxonomy—predicting and writing test questions from book and notes
    • Group study
    • Individual study
    • Concept cards
    • Study guides
    • Created concept maps of the chapters
    • Synthesized notes
    • Other methods?

Students are given 1-3 minutes to reflect in journal writing upon those questions. They are then prompted to analyze where the test questions came from (book, notes, power point, lab, supplemental essay, online materials, etc.) It may be helpful to have students work collaboratively for this.     

An Analysis of the Test—Where the Information Came From

  • For each question identify where the test question came from:
    • Book (B)
    • In-class notes (C)
    • Online materials (O)
    • Supplemental readings (S)
    • Not sure (?)

After identifying where the test information came from, students are then prompted to reflect in journal writing upon the questions they missed and how they might study differently based upon the questions they missed and where the questions came from. For example, a student may realize that he or she missed all of the questions that came from the book. That student may then make a goal to synthesizing class notes right after class with material from the book 30 minutes after class, and then use note reduction to create a concept map to study for the next test.

Another student might realize that he or she missed questions because of test-taking errors. For example, she didn’t carefully read the entire question and then chose the wrong response. To resolve this issue, she decided she would underline question words on the test and in attempt to slow down while reading test questions. She also realized that she changed several responses that she had correct. She will resist the urge to overthink her choices and change responses on the next test.

Next, students are taught about Bloom’s Taxonomy and how it is used by professors to write exams. In small groups, students then use Bloom’s Taxonomy to identify question types. This will take about 20-30 minutes—depending upon the length of the test. For example, students would identify the following test question as a comprehension-level question: Which of the following best describes positive reinforcement? Whereas, the following question would be noted as an application-level question: Amy’s parents give her a lollipop every time she successfully uses the toilet. What type of reinforcement is this?

Question Type: Identify What Level of Bloom’s Taxonomy the Test Question is Assessing

  • Knowledge-level questions
  • Comprehension
  • Application
  • Analysis
  • Synthesis
  • Evaluation

Students sometimes struggle with distinguishing the different levels of questions. So, it is helpful to also ask small groups to share their identified questions with the large group, as well as how they determined it to be that level of question. The professor also is a helpful resource in this discussion.

After discussion of the questions types, students then return to individual reflection, as they are asked to count the number of questions they missed for each level of Bloom’s Taxonomy. They are also asked to reflect upon what new strategies they will use to study based on this new awareness.

Adding It All Up

  • Count the number of questions missed in each level of Bloom’s Taxonomy.
  • Which types of questions did you miss most often?
  • Compare this with your study methods.
  • What adjustments might you make in your studying and learning of class material based on this information? Which levels of Bloom’s Taxonomy do you need to focus more on with your studying?

Finally, students are asked to use the class reflections and post-test assessment to create a new learning plan for the course. (See the learning plan in my previous post, Facilitating Metacognition in the Classroom: Teaching to the Needs of Your Students). Creating the Learning Plan could be a graded assignment that students are asked to do outside of class and then turn in. Students could also be referred to the Academic Resource Center on campus for additional support in formulating the Learning Plan. Additionally, a similar post-test assessment could be assigned outside of class for subsequent exams and be assigned a point value. This would allow for ongoing metacognitive reflection and self-regulated learning.

This type Cognitive Strategy Instruction (Scheid, 1993) embedded into the classroom offers students a chance to become more aware of their own cognitive processes, strategies for improving learning, and the practice of using cognitive and metacognitive processes in assessing their success (Livingston, 2003). Importantly, these types of reflective assignments move students away from a fixed mindset to a growth mindset (Dweck, 2006). As Masters (2013) pointed out “Assessment information of this kind provides starting points for teaching and learning.” Additionally, because post-test assessment offers students greater self-efficacy, control of their own learning, purpose, and an emphasis on the learning rather than the test score, it also positively affects motivation (VanZile-Tamsen & Livingston, 1999).

References

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Balantine Books.

Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13 (4), 277-287. Retrieved from https://www.researchgate.net/publication/232452693_Metacognition_Defining_Its_Facets_ad_Levels_of_Functioning_in_Relation_to_Self-Regulation_and_Co-regulation

Livingston, J. A. (2003). Metacognition: An overview. Retrieved from https://files.eric.ed.gov/fulltext/ED474273.pdf

Masters, G. N. (2013). Towards a growth mindset assessment. Retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?article=1017&context=ar_misc

Palmer, D. J., & Goetz, E. T. (1988). Selection and use of study strategies: The role of studier’s beliefs about self and strategies. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 41-61). San Diego, CA: Academic.

Palmer, D. J., & Goetz, E. T. (1988). Selection and use of study strategies: The role of studier’s beliefs about self and strategies. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 41-61). San Diego, CA: Academic.

Pintrich, P. R., & DeGroot, E. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40

Palmer, D. J., & Goetz, E. T. (1988). Selection and use of study strategies: The role of studier’s beliefs about self and strategies. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 41-61). San Diego, CA: Academic.

Pintrich, P. R., & DeGroot, E. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40

Pintrich, P. R., & DeGroot, E. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.

VanZile-Tamsen, C. & Livingston, Jennifer. J. A. (1999). The differential impact of motivation on the self regulated strategy use of high- and low-achieving college student. Journal of College Student Develompment, (40)1, 54-60. Retrieved from https://www.researchgate.net/publication/232503812_The_differential_impact_of_motivation_on_the_self-regulated_strategy_use_of_high-_and_low-achieving_college_students

Vogeley, K., Kurthen, M., Falkai, P., & Maier, W. (1999). Essential functions of the human self model are implemented in the prefrontal cortex. Consciousness and Cognition, 8, 343-363.


Teaching Study Skills Through Retrieval Practice, Interleaving, and Spaced Practice

by Dana Melone, Cedar Rapids Kennedy High School

We all want our students to study effectively. Many of our students fall into the trap of doing what they have always done. They do this for a lot of reasons. Maybe it has worked for them in the past and now suddenly their method is not working, maybe they were never taught how to study and assumed it must be intuitive, or maybe they have never needed to study in the past. The list can go on and on. As educators, we know that study habits are an important part of academic achievement.

I teach five sections of Advanced Placement Psychology with an average of about 150 students per school year. These students are sophomores, juniors, and seniors from a variety of different socioeconomic, racial and ethnic groups. My students also come in with a variety of experience in taking advanced courses and a variety of ranges of grade point averages. I talk with my students at the beginning of the school year about their study habits to determine whether I need to teach them study skills.

The words "Patterns of Misconceptions" are repeated several times

Here are just a few of the patterns of studying/learning misconceptions that I see:

Misconceptions
  • More time studying means better knowledge of the topic
  • Re-reading the content or re-reading the notes is effective studying
  • Re-examining course slides is effective studying
  • Flashcards of vocabulary will lead to exam success
  • Listening to music or watching TV while studying is ok
  • Using books and notes while studying is taking place is enough
  • Studying looks the same no matter the course or style of exam
  • Every item in the course must be studied

After seeing these patterns year after year, I wanted to teach my students effective study methods and the use of metacognition in studying. I wanted the focus to be on continuous modeling of this in the course and honest discussions about how this can be used at home as a study tool. For the past few years I have been focusing on clearing up misconceptions about studying, and teaching students methods of retrieval practice, interleaving multiple areas of content, and spacing out their retrieval practice over time for better retention. This focus helps my students provide their own feedback on what they know and do not know.

Retrieval practice is defined as “A learning strategy where we focus on getting information out” (retrievalpractice.org). This means I am teaching my students to talk, write, and answer questions about the content without looking at the material while they are doing it. By focusing on on retrieval over time (spacing of learning), plus weaving in multiple topics from the course at a time (interleaving), I am helping my students become self sufficient in determining what they know and what they do not know about the content and its connection to other units in the course.

I incorporate these three high efficacy learning strategies in my classroom in multiple ways and encourage students to use similar techniques in their own studying. Here are a few examples:

  • At the end of every unit I provide my students with an online questionnaire that sends them their results and results to me as well. This questionnaire poses questions asking them to state as much information about a current topic as they possibly can, to then connect that information to content from each of our previous units. They then reflect on what they remembered and how well they could connect it to other content areas. They are supposed to answer the questionnaire without referring to notes or text materials.
  • I provide students with multiple questions from a chapter and have them answer them in as much detail as they can without looking at their notes. They then change ink color and use books and notes answer what they could not originally answer on their own. This exercise provides them with a great visual of what they did and did not know for an entire unit. I call this a brain dump. This name is based off others I have seen in my professional networks. These can be used at any time in a unit, but I prefer to give these to students a week or so prior to an exam.
  • Another activity similar to a brain dump is a single topic retrieval activity. I take a topic that is addressed over multiple content areas and have them write down as much content as they can without their books or notes by applying it to each area it has appeared in. They then switch ink colors and walk around the room to fill in their gaps. This often includes a prediction of how it might appear in future topics. I call this a retrieval practice challenge. This activity reminds students that we have discussed this topic before from a different viewpoint or lense and allows them to have some predictive discussion of a new lens.

            My students have really responded to these methods but it has taken a lot of teaching and reteaching the method and the reasoning behind the use of the method. Many of them have started creating them for their other classes. I have seen a decrease in students asking me for fill in the blank study guides and an increase in students asking me for tips on how to use retrieval practice to study a variety of topics. My plan moving forward is to show students correlation data from their exams showing the relationship between retrieval practice and test scores.


Being Authentic: Modeling Metacognitive Growth and Connecting with Students

by Patrick Cunningham, Ph.D., Rose-Hulman Institute of Technology

Learning takes effort and can feel hard at times. Likewise, learning to manage one’s learning processes better, growing metacognitively, is an effortful process. Because approaches to learning are habitual, deeply ingrained over time, changing them is hard. Metacognitive growth often progresses in fits and starts with cycles improvement and relapse into old habits. This is true for our students and for us as instructors. Remembering this can help us help them better with metacognitive lessons that can guide them throughout their lives.

I often say that teaching students about metacognition and engaging them in their metacognitive growth is one of the most important and authentic things I do academically. I have expertise within my field (Mechanical Engineering, Dynamic Systems and Control) and I am accomplished at applying it to engineering problems. When I engage students in learning this content, I model this expertise. I strive to make my content-focused teaching authentic, but it isn’t authentic in the same way as providing instructional experiences for my students on metacognition. While I may know which disciplinary concepts students will likely struggle with, my struggles with those concepts are a distant memory. However, as I engage my students in metacognitive growth, i.e., changing my habitual approaches to learning, my struggles are concurrent with theirs.

image of a human figure helping another human figure up a hill

Becoming a student again

For example, I have encountered my metacognitive struggles as I have been learning German, initially as a hobby and now more intentionally as I prepare for a sabbatical in Germany. About a year-and-a-half ago I decided to sign up for a Duolingo account to see if I could resurrect and build on my prior German language experience. I had just signed my children up for accounts to practice Spanish and sensed an opportunity to finally work towards a long-held personal goal – becoming proficient in another language. Armed with my knowledge of metacognition, I wanted to make my practice effective and efficient. I set a modest but good practice goal of two grammar modules per day, and I jumped in. I have been consistent in my daily practice – only missing about three days in 18 months – and I have added some varied strategies to practice German – Duolingo stories (spoken dialogues with text), trying to use basic phrases in my daily life, and trying to read German news stories. However, I have also noticed some metacognitive pitfalls in my language practice, even as I have gotten more serious about it.

I have not engaged in recall practice, despite the pop-up messages reminding me that I should take time after a session to recall new vocabulary and key grammar and usage insights. I also know its value, but I tell myself I just don’t have time. I am avoiding a good strategy and making poor use of my metacognitive knowledge of strategies and tasks.

I also lack a clear learning goal. What does proficient mean? How will I know I have achieved it? What are appropriate incremental goals that build towards proficiency? Admittedly, for most of this time, this has been more of a hobby pursuit, but if I really want to develop the skill, I need more specific, even if still modest, learning goals. This is poor metacognitive planning.

Then there is how I track my progress. I am sad to say I have taken pride in the number of XP points (virtual points within Duolingo) I have accrued and the number of modules I have crossed off. They are easy to count, but this does not really assess my proficiency with the German language accurately. This is poor metacognitive monitoring of my learning.

Within the story modules, I am tentative and find myself relying too much on being able to hover over the words to see the definition before answering the comprehension questions. Why do I find it hard to commit to an answer, right or wrong, and learn from it? It feels hard, but as I tell my students, this is how you know you are learning. I am relying on a lower quality strategy because it feels good.

Acknowledging the Same Struggles

So, each time I bring up metacognition with my students, I am faced with the reality of my struggles with it, as demonstrated by the pitfalls in my German language practice. Thus, I teach about metacognition, not as an expert who has it all figured out, but rather as one who is, perhaps, further along the path. This is humbling. So, what can I do with this struggle? Can I really engage my students in their metacognitive growth if I am struggling with my own metacognitive growth? Yes! But how I go about it matters.

If I ignore talking about metacognition altogether, then I might avoid feeling uncomfortable about my failings with it. But I would not likely grow myself nor help my students grow metacognitively. If I pretend I have it all figured out, then I risk being found-out, losing my credibility, and sabotaging my students’ potential metacognitive growth.

However, if embrace my struggles with metacognition and am honest about them with my students, then I might be able to grow myself while I also help my students. There are at least two mechanisms for positive impact, acknowledging my similarity to my students and providing my students a concrete model of metacognitive growth in practice.

When I accept that I am like my students and my students are like me in struggling with metacognitive practice and growth (e.g., my German language studies), I gain a more complete view of my students. I can no longer view them as just lazy, unmotivated, or lacking in work ethic. Instead I can see sincere effort and a desire to learn and do well in classes, in spite of less effective learning behaviors. When I see my students in this way, I have a better attitude when I interact with them. This enables me to authentically praise the positive aspects of their learning behavior and to more gently challenge the less effective aspects they are relying on. It can move our interaction from a place of discouragement to one of encouragement and can help students to view their ability to grow into the learning challenge before them. When I recognize that my students have similar barriers to metacognitive growth as I do, I am able to be more compassionate and supportive as I help them face the challenges they experience in their learning.

Points of Connection

Sharing my own metacognitive struggles, e.g., with my German language practice, can provide a point of connection with my students. Students can have a hard time identifying with their instructors, viewing us as experts with experiences far removed from theirs. It is heartening to see students warm up to me and talk more openly and honestly about their approaches to learning when I have shared elements of my struggles with learning German. Suddenly, the relational distance between us shrinks because I have a present learning experience, concurrent with theirs, that they can readily identify with. Such authentic connections build trust and a foundation for a relationship, which can lead to further support and processing of their learning experiences.

Beyond forging an authentic connection, I can also constructively model of the effortful and continual path to becoming a more skillful learner, i.e., metacognitive growth. Sharing my critiques of my German language practice can demonstrate metacognitive evaluation of my learning processes and my openness to ongoing refinement. However, metacognitive growth does not stop with recognizing ineffective or less effective learning strategies. It requires doing something about it, enacting a productive change. For example, I could share a more refined and specific goal – such as, wanting to be able to engage in pleasantries and make small talk in German – which also helps direct my practice and how I monitor my progress.

How can this look within a specific class? This winter I have been teaching computer programming, and I have found it useful with a few students to draw analogies between my language practice and working towards proficiency with programming concepts. Becoming conversant in German requires more than just knowing German grammar rules, e.g., declensions for accusative cases. I must practice using it in conversation, that is, applying it. Similarly, to become proficient with programming I must know how to write conditional statements and loops, but I also must know how to apply them in various ways to accomplish a specified task – I have to practice applying the concepts. I was trying to model the need to align learning strategies with specific learning goals.

In summary, if I can identify with my students, I can better help them with their metacognitive growth. It helps me to be more gentle and supportive in my desire to see them grow metacognitively. It can also can help my students connect with me and see that they too can persist in their metacognitive growth when I am forthright with my metacognitive shortcomings. Being authentic matters and it can help me do what I think is the most important teaching that I do, helping my students become better learners. These are enduring lessons that can help them be successful throughout their lives, even if they forget the content that I was teaching!


Metacognitive Self-Assessment, Competence and Privilege

by Steven Fleisher, Ph.D., California State University Channel Islands

Recently I had students in several of my classes take the Science Literacy Concept Inventory (SLCI) including self-assessment (Nuhfer, et al., 2017). Science literacy addresses one’s understanding of science as a way of knowing about the physical world. This science literacy instrument also includes self-assessment measures that run parallel with the actual competency measures. Self-assessment skills are some of the most important of the metacognitive competencies. Since metacognition involves “thinking about thinking,” the question soon becomes, “but thinking about what?”

Dunlosky and Metcalfe (2009) framed the processes of metacognition across metacognitive knowledge, monitoring, and control. Metacognitive knowledge involves understanding how learning works and how to improve it. Monitoring involves self-assessment of one’s understanding, and control then involves any needed self-regulation. Self-assessment sits at the heart of metacognitive processes since it sets up and facilitates an internal conversation in the learner, for example “Am I understanding this material at the level of competency needed for my upcoming challenge?” This type of monitoring then positions the learner for any needed control or self-regulation, for instance “Do I need a change my focus, or maybe my learning strategy?” Further, self-assessment is affective in nature and is central to how learning works. From a biological perspective, learning involves the building and stabilizing of cognitive as well as affective neural networks. In other words, we not only learn about “stuff”, but if we engage our metacognition (specifically self-assessment in this instance), we are enhancing our learning to include knowing about “self” in relation to knowing about the material.

This Improve with Metacognition posting provides information that was shared with my students to help them see the value of self-assessing and for understanding its relationship with their developing competencies and issues of privilege. Privilege here is defined by factors that influence (advantage or disadvantage) aggregate measures of competence and self-assessment accuracy (Watson, et al., 2019). Those factors involved: (a) whether students were first-generation college students, (b) whether they were non-native English-language students, and (c) whether they had an interest in science.

The figures and tables below result from an analysis of approximately 170 students from my classes. The narrative addresses the relevance of each of the images.

Figure 1 shows the correlation between students’ actual SLCI scores and their self-assessment scores using Knowledge Survey items for each of the SLCI items (KSSLCI). This figure was used to show students that their self-assessments were indeed related to their developing competencies. In Figure 2, students could see how their results on the individual SLCI and KSSLCI items were tracking even more closely than in Figure 1, indicating a fairly strong relationship between their self-assessment scores and actual scores.

scatterplot graph of knowledge survey compared to SCLI scores
Figure 1. Correlation with best-fit line between actual competence measures via a Science Literacy Concept Inventory or SLCI (abscissa) and self-assessed ratings of competence (ordinate) via a knowledge survey of the inventory (KSSLCI) wherein students rate their competence to answer each of the 25 items on the inventory prior to taking the actual test.
scatter plot of SCLI scores and knowledge survey scores by question
Figure 2. Correlation with best-fit line between the group of all my students’ mean competence measures on each item of the Science Literacy Concept Inventory (abscissa) and their self-assessed ratings of competence on each item of the knowledge survey of the inventory (KSSLCI).

Figure 3 demonstrates the differences in science literacy scores and self-assessment scores among their different groups as defined by the number of science courses taken. Students could readily see the relationship between the number of science courses taken and improvement in science literacy. More importantly in this context, students could see that these groups had a significant sense of whether or not they knew the information, as indicated by the close overlapping of each pair of green and red diamonds. Students learn that larger numbers of participants can provide more confidence to where the true means actually lies. Also, I can show the meaning of variation differences within and between groups. In answering questions about how we know that more data would clarify relationships, I bring up an equivalent figure from our national database that shows the locations of the means within 99.9% confidence and the tight relationship between groups’ self-assessed competence and their demonstrated competence.

categorical plot by number of college science courses completed
Figure 3. Categorical plot of my students in five class sections grouped by their self-identified categories of how many college-level science courses that they have actually completed. Revealed here are the groups’ mean SLCI scores and their mean self-assessed ratings. Height of the green (SLCI scores) and red (KSSLCI self-assessments) diamonds reveals with 95% confidence that the actual mean lies within these vertical bounds.

Regarding Figure 4, it is always fun to show students that there’s no significant difference between males and females in science literacy competency. This information comes from the SLCI national database and is based on over 24,000 participants.

categorical plot by binary gender
Figure 4. Categorical plot from our large national database by self-identified binary gender categories shows no significant difference by gender in competence of understanding science as a way of knowing.

It is then interesting to show students in that, in their smaller sample (Figure 5), there is a difference between the science literacy scores of males and females. The perplexed looks on their faces are then addressed by the additional demographic data in Table 1 below.

categorical plot by binary gender for individual class
Figure 5. Categorical plot of just my students by binary gender reveals a marginal difference between females and males, rather than the gender-neutral result shown in Fig. 4.

In Table 1, students could see that higher science literacy scores for males in their group were not due to gender, but rather, were due to significantly higher numbers of English as a non-native language for females. In other words, the women in their group were certainly not less intelligent, but had substantial, additional challenges on their plates.  

Table 1: percentages of male and female students as first generation, English and non-native speaker, and with respect to self-report interest to major in science

Students then become interested in discovering that the women demonstrated greater self-assessment accuracy than did the men, who tended to overestimate (Figure 6). I like to add here, “that’s why guys don’t ask for directions.” I can get away with saying that since I’m a guy. But more seriously, I point out that rather than simply saying women need to improve in their science learning, we might also want to help men improve in their self-assessment accuracy.   

categorical plot by gender including self-assessment data
Figure 6. The categorical plot of SLCI scores (green diamonds) shown in Fig. 5 now adds the self-assessment data (red diamonds) of females and males. The trait of females to more accurately self-assess that appears in our class sample is also shown in our national data. Even small samples taken from our classrooms can yield surprising information.

In Figure 7, students could see there was a strong difference in science literacy scores between Caucasians and Hispanics in my classes. The information in Table 2 below was then essential for them to see. Explaining this ethnicity difference offers a wonderful discussion opportunity for students to understand not only the data but what it reveals is going on with others inside their classrooms.

Figure 7. The categorical plot of SLCI scores by the two dominant ethnicities in my classroom. My campus is a Hispanic Serving Institution (HSI). The differences shown are statistically significant.

Table 2 showed that the higher science literacy scores in this sample were not simply due to ethnicity but were impacted by significantly greater numbers of first-generation students and English as a non-native language between groups. These students are not dumb but do not have the benefits in this context of having had a history of education speak in their homes and are navigating issues of English language learning. 

Table 2: percentage of white and hispanic students who report to be first generation students, English as non-native speakers, and interested in majoring in science.

When shown Figure 8, which includes self-assessment scores as well as SLCI scores, students were interested to see that both groups demonstrated fairly accurate self-assessment skills, but that Hispanics had even greater self-assessment accuracy than their Caucasian colleagues. Watson et. al (2019) noted that strong self-assessment accuracy for minority groups comes about from a need for being understandably cautious.

categorical plot by ethnicity and including self-assessment
Figure 8. The categorical plot of SLCI scores and self-assessed competence ratings for the two dominant ethnicities in my classroom. Groups’ collective feelings of competence, on average, are close to their actual competence. Explaining these results offered a wonderful discussion opportunity for students.

Figure 9 shows students that self-assessment is real. In seeing that most of their peers fall within an adequate range of self-assessment accuracy (between +/- 20 percentage points), students begin to see the value of putting effort into developing their own self-assessment skills. In general, results from this group of my students are similar to those we get from our larger national database (See our earlier blog post, Paired Self-Assessment—Competence Measures of Academic Ranks Offer a Unique Assessment of Education.)

distribution of self-assessment accuracy for individual course
Figure 9. The distribution of self-assessment accuracy of my students in percentage points (ppts) as measured by individuals’ differences between their self-assessed competence by knowledge survey and their actual competence on the Concept inventory.

Figure 10 below gave me the opportunity to show students the relationship between their predicted item-by-item self-assessment scores (Figure 9) and their postdicted global self-assessment scores. Most of the scores fall between +/- 20 percentage points, indicating good to adequate self-assessment. In other words, once students know what a challenge involves, they are pretty good at self-assessing their competency.

distribution of self-assessment accuracy for individual course after taking SCLI
Figure 10. The distribution of self-assessment accuracy of my students in percentage points (ppts) as measured by individuals’ differences between their postdicted ratings of competence after taking the SLCI and their actual scores of competence on the Inventory. In general, my students’ results are similar in self-assessment measured in both ways.

In order to help students further develop their self-assessment skills and awareness, I encourage them to write down how they feel they did on tests and papers before turning them in (postdicted global self-assessment). Then they can compare their predictions with their actual results in order to fine-tune their internal self-assessment radars. I find that an excellent class discussion question is “Can students self-assess their competence?” Afterward, reviewing the above graphics and results becomes especially relevant. We also review self-assessment as a core metacognitive skill that ties to an understanding of learning and how to improve it, the development of self-efficacy, and how to monitor their developing competencies and control their cognitive strategies.

References

Dunlosky, J. & Metcalfe, J. (2009). Metacognition. Sage Publications Inc., Thousand Oaks, CA.

Nuhfer, E., Fleisher, S., Cogan, C., Wirth, K., & Gaze, E. (2017). How Random Noise and a Graphical Convention Subverted Behavioral Scientists’ Explanations of Self-Assessment Data: Numeracy Underlies Better Alternatives. Numeracy, Vol 10, Issue 1, Article 4. DOI: http://dx.doi.org/10.5038/1936-4660.10.1.4

Watson, R., Nuhfer, E., Nicholas Moon, K., Fleisher, S., Walter, P., Wirth, K., Cogan, C., Wangeline, A., & Gaze, E. (2019). Paired Measures of Competence and Confidence Illuminate Impacts of Privilege on College Students. Numeracy, Vol 12, Issue 2, Article 2. DOI: https://doi.org/10.5038/1936-4660.12.2.2


The Evolution of Metacognition in Biological Sciences

By Lindsay Doukopoulos, Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning at Auburn University, and blog mini-series editor.

Much of the literature on metacognition focuses on strategies that faculty can use to improve metacognitive skills in their students and the benefits of such skills. Our mini-series tackles a different kind of problem: how can a department redesign its curriculum to improve metacognition for all students and how will it know if improvement has actually occurred?  We believe our efforts can inform others across a variety of disciplines.

Our answer to this question takes the form of a case study in five parts about our collaborative and ongoing efforts to redesign the Department of Biological Sciences’ undergraduate curriculum and program assessment with a goal of improving metacognition for its students and demonstrating that improvement with data. We use a narrative structure to present the key inflection points in this process as well as lessons learned and best practices from our diverse perspectives.

Our collaborators include: Associate Dean for Academic Affairs for the College of Sciences and Mathematics, Bob Boyd (also a Biological Sciences professor and formerly the department’s Undergraduate Program Officer); Associate Director of Academic Assessment, Katie Boyd; Associate Director of the Office of University Writing, Chris Basgier; Chair of the Department of Biological Sciences, Scott Santos; and Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning, Lindsay Doukopoulos.  

This timeline provides an overview of our efforts while our individual posts go into more detail about specific strategies and outcomes:  

Ideation: 

June 2016: Department leaders attend PULSE Institute and decide to make metacognition a student learning outcome (SLO) for all undergraduate programs the Department of Biological Sciences (hereafter, Biology) 

May 2017: Program assessment reports at this time include only two student learning outcomes (metacognition not one of them) for each of the three undergraduate programs in Biology 

August 2017: Faculty retreat led by NSF Vision & Change experts introducing metacognitive teaching strategies  

Commitment: 

October 2017: Learning Improvement Initiative launched by Biggio Center and Office of Academic Assessment: Biology proposes to improve SLO 6 – Metacognition  

Spring 2018: Biology’s curriculum committee develops a plan for improvement and creates an ideal (“aspirational”) curriculum map to share at the 2018 fall faculty retreat 

Lindsay Doukopoulos leading faculty development on metacognition at the 2018 Biology Faculty Retreat
Lindsay Doukopoulos leading faculty development on metacognition at the 2018 Biology Faculty Retreat

Conflict: 

August 2018: Faculty retreat, aka “Metacognition Massacre” – widespread faculty rejection of the metacognition SLO on the curriculum map 

A New Approach: 

Fall 2018: A three-part workshop series created by Office of University Writing (OUW) and the Biggio Center leads faculty to redefine the metacognition SLO and introduces strategies to support faculty teaching  

Turning Point:  

December 2018: Outcomes of the workshop series, including the new definition of SLO 6, are presented at a faculty meeting and the faculty vote to approve the new definition  

Assessing Metacognition:  

January – April 2018: Office of Academic Assessment and the Biggio Center lead Biology’s curriculum committee in creating a metacognitive questionnaire for graduating students and a rubric to assess the level of metacognition evidenced in the responses 

Improving Metacognition: 

Summer 2019: Biology invests in comprehensive strategy to promote metacognition across the curriculum using ePortfolios and several faculty participate in an intensive course redesign program 

What now?  

Fall 2019: OUW and Biggio provide ongoing support of teaching interventions to improve metacognition; Office of Academic Assessment provides ongoing support of the assessment of this work 

What’s next? 

Spring 2020: Gather baseline data on graduates’ metacognitive capabilities  Goals: Based on our efforts and an ongoing collection of data, we expect to see increases in students’ metacognitive abilities over time 


How do you know you know what you know?

by Patrick Cunningham, Ph.D., Rose-Hulman Institute of Technology

Metacognition involves monitoring and controlling one’s learning and learning processes, which are vital for skillful learning. In line with this, Tobias and Everson (2009) detail the central role of accurate monitoring in learning effectively and efficiently. Metacognitive monitoring is foundational for metacognitive control through planning for learning, selecting appropriate strategies, and evaluating learning accurately (Tobias & Everson, 2009).

Hierarchy of Metacognitive Control, with Monitoring Knowledge at the bottom, followed by Selecting Strategies, Then Evaluating Learning, with Planning at the top

Figure 1 – Hierarchy of metacognitive regulatory processes. Adapted from Tobias and Everson (2009).

Unfortunately, students can be poor judges of their own learning or fail to engage in the judging of their learning and, therefore, often fail to recognize their need for further engagement with material or take inappropriate actions based on inaccurate judgements of learning (Ehrlinger & Shain, 2014; Winne and Nesbit, 2009). If a student inaccurately assesses their level of understanding, they may erroneously spend time with material that is already well known or they may employ ineffective strategies, such as a rehearsal strategy (e.g., flash cards) to build ROTE memory when they really need to implement an elaborative strategy (e.g., explaining the application of concepts to a new situation) to build richer integration with their current knowledge. This poor judgement extends to students’ perceptions of the effectiveness of their learning processes, as noted in the May 14th post by Sabrina Badali, Investigating Students’ Beliefs about Effective Study Strategies[. There Badali found that students were more confident in using massed practice over interleaved practice even though they performed worse with massed practice.

Fortunately, we can help our students to develop more accurate self-monitoring skills. The title question is one of my go-to responses to student claims of knowing in the face of poor performance on an assignment or exam. I introduced it in my April 4th blog post, Where Should I Start with Metacognition? It gently, but directly asks for evidence for knowing. In our work on an NSF grant to develop transferable tools for engaging students in their metacognitive development, my colleagues and I found that students struggle to cite concrete and demonstrable (i.e., objective) evidence for their learning (Cunningham, Matusovich, Hunter, Blackowski, and Bhaduri, 2017). It is important to gently persist. If a student says they “reviewed their notes” or “worked many practice problems,” you can follow up with, “What do you mean by review your notes?” or “Under what conditions were you working the practice problems?” The goal is to learn more about the students’ approach while avoiding making assumptions and helping the student discover any mismatches.

We can also spark monitoring with pedagogies that help students accurately uncover present levels of understanding (Ehrlinger & Shain, 2014). Linda Nilson (2013) provides several good suggestions in her book Creating Self-Regulated Learners. Retrieval practice takes little time and is quite versatile. Over a few minutes a student recalls all that they can about a topic or concept, followed by a short period of review of notes or a section of a book. The whole process can be done individually, or as individual recall followed by pair or group review. Things that are well-known are present with elaborating detail on the list. Less well-known material is present, but in sparse form. Omissions indicate significant gaps in knowledge. The practice is effortful, and students may need encouragement to persist with it.

I have used retrieval practice at the beginning of classes before continuing on with a topic from the previous day. It can also be employed as an end-of-class summary activity. I think the value added is worth the effort. Because of its benefits and compactness, I also encourage students to use retrieval practice as a priming activity before regular homework or study sessions. Using it in class can also lower students’ barriers to using it on their own, because it makes it more familiar and it communicates the value I place on it.

Nilson (2013) also offers “Quick-thinks” and Think Aloud problem -solving. “Quick-thinks” are short lesson breaks and can include “correct the error” in a short piece of work, “compare and contrast”, “reorder the steps”, or other activities. A student can monitor their understanding by comparing to the instructor’s answer or class responses. Think Aloud problem-solving is a pair activity where one student talks through their problem-solving process while the other student listens and provides support, when needed, for example, by prompting the next step or asking a guiding question. Students take turns with the roles. A student’s fluency in solving the problem or providing support indicates deeper learning of the material. If the problem-solving or the support are halting and sparse, then those concepts are less well-known by the student. As my students often study in groups outside of class, I recommend that they have the person struggling with a problem or concept talk through their thinking out loud while the rest of the group provides encouragement and support.

Related to Think Alouds, Chiu and Chi (2014) recommend Explaining to Learn. A fluid explanation with rich descriptions is consistent with deeper understanding. A halting explanation without much detail uncovers a lack of understanding. I have used this in various ways. In one form, I have one half of the class work one problem and the other half work a different problem or a variant of the first. Then I have them form pairs from different groups and explain their solutions to one another. Both students are familiar with the problems, but they have a more detailed experience with one. I also often use this as I help students in class or in my office. I ask them to talk me through their thinking up to the point where they are stuck, and I take the role of the supporter.

The strategies above provide enhancements to student learning in their own right, but they also provide opportunities for metacognitive monitoring – checking their understanding against a standard or seeking objective evidence to gauge their level of understanding. To support these metacognitive outcomes I make sure to explicitly draw students’ attention to the monitoring outcomes when I use pedagogies to support monitoring. I am also transparent about this purpose and encourage students to seek better evidence on their own, so they can truly know what they know.

As you consider adding activities to your course that support accurate self-assessment and monitoring, please see the references for further details. You may also want to check out Dr. Lauren Scharff’s post “Know Cubed” – How do students know if they know what they need to know? In this post Dr. Scharff examines common causes of inaccurate self-assessment and how we might be contributing to it. She also offers strategies we can adopt to support more accurate student self-assessment. Let’s help our student generate credible evidence for knowing the material, so they can make better choices for their learning!

References

Chiu, J. L. & Chi, M. T. H.  (2014). Supporting Self-Exlanation in the Classroom. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.). Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php

Cunningham, P., & Matusovich, H. M., & Hunter, D. N., & Blackowski, S. A., & Bhaduri, S. (2017), Beginning to Understand Student Indicators of Metacognition.  Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. https://peer.asee.org/27820

Ehrlinger, J. & Shain, E. A.  (2014). How Accuracy in Students’ Self Perceptions Relates to Success in Learning. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.). Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php

Nilson, L. B. (2013). Creating Self-Regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills. Stylus Publishing: Sterling, VA.

Tobias, S. & Everson, H. (2009). The Importance of Knowing What You Know: A Knowledge Monitoring Framework for Studying Metacognition in Education. In Hacker, D., Dunlosky, J., & Graesser, A. (Eds.) Handbook of Metacognition in Education. New York, NY: Routledge, pp. 107-127.

Winne, P. & Nesbit, J. (2009). Supporting Self-Regulated Learning with Cognitive Tools. In Hacker, D., Dunlosky, J., & Graesser, A. (Eds.) Handbook of Metacognition in Education. New York, NY: Routledge, pp. 259-277.


Changing Campus Culture with the Ace-Your-Course Challenge

In the final post of the guest blog series on “Working with Faculty to Promote Metacognition,” Dr. Eric Kaldor discusses lessons learned from the implementation of a campus-wide metacognition program inspired by Saundra McGuire’s work. The associated research project was awarded the Robert J. Menges Award for Outstanding Research in Educational Development by the Professional and Organizational Development (POD) Network.

by Eric Kaldor, Ph.D.; Associate Director, Sheridan Center for Teaching & Learning, Brown University

For many faculty members, the “fact” that some students are just not capable of college-level learning remains part of the taken-for-granted assumptions embedded in the culture of disciplines and campuses. Despite significant efforts to share insights from the scholarship on metacognition and growth mindsets (e.g. Doyle & Zakrajsek, 2013; Dweck, 2016; McGuire, 2015; Nilson, 2013), campus cultures are slow to change, and fixed mindsets continue to dominate many institutions. This post describes efforts to change the culture at the University of Rhode Island, the communication strategy we used, and some lessons learned.

With approximately 14,000 undergraduate students and 1,000 full and part-time faculty, the University of Rhode Island is a challenging setting to advance culture change. Our story began with a conversation with Melvin Wade, former Director of the Multicultural Student Services Center (MSSC). I was working in the Office for the Advancement of Teaching & Learning (ATL) and planning for Saundra McGuire to visit our campus. I was particularly concerned to fill our 1,000-person auditorium with students for her “Metacognition is Key” workshop. When I asked for his advice, Melvin insisted we must ensure her visit had a lasting impact on our campus. Toward this end, we assembled a group of professional staff and graduate students from ATL, the MSSC, the Academic Enhancement Center, First-Year Programs, and Professional Advising. Over a series of conversations, this informal group conceived of something we came to call the Ace-Your-Course (AYC) Challenge. We assumed we would only run the AYC Challenge once as a companion to Dr. McGuire’s workshop. Instead, a snowstorm gave the Challenge a much longer life.

Building on the McGuire Model

We designed the AYC Challenge to extend students’ metacognitive experience and reflections beyond Dr. McGuire’s workshop. We developed the AYC Challenge as four weekly self-assessment surveys (for detailed description see Kaldor & Swanson, 2019) to create additional metacognitive experiences (Flavell, 1979) by encouraging students to:

  1. Test learning strategies relevant to them individually.
  2. Engage in key practices for metacognitive reflection: observation, description, evaluation, and action planning.
  3. Feel part of a larger community working to grow as learners.

When a snowstorm postponed Dr. McGuire’s visit to the next semester, our multi-unit team led her workshop twice using slides and talking points from her book (McGuire, 2015) and invited students to participate in the AYC Challenge. Of the 240 students attending a workshop, 50 completed all four weeks of the challenge. After we shared the positive results from our pilot with faculty members, many encouraged their students to attend Dr. McGuire’s rescheduled workshop in September 2017. Some went further and agreed to share grade data as part of an IRB-approved study to examine how participation affected grades. We specifically identified a set of gateway science courses from Chemistry, Biology, and Nutrition and Food Sciences that have large enrollments of first-year students.

Over 1,000 students attended Dr. McGuire’s workshop with some in remote viewing locations, and 202 of those completed the second AYC Challenge. The self-reported results for this larger group were strikingly similar to those from students in the pilot AYC Challenge when we led the workshops. Holly Swanson and I analyzed final grades for 979 students in the eight gateway science sections (347 attended the workshop and of those 55 completed the challenge) using OLS regression with controls for several predictors of academic performance including high school GPA and exam 1 z-score. Compared with their peers who did not attend the workshop or participate in the challenge, attending the workshop and completing the AYC Challenge was associated with a final course grade half of a letter grade higher (Kaldor & Swanson, 2018).

Inclusive and Extensive Communication

Much of our success originated from a spiral of communication that grew outwards from a core group of professional staff and graduate students who became involved in planning for Dr. McGuire’s originally scheduled visit. Our colleagues working in various student support services helped develop a plan to reach students and motivate them to attend the workshop and participate in the challenge. These colleagues advised us on when to hold the workshop, how to market our efforts, and what kinds of messages would appeal to students.

One critical piece of advice was that students were more likely to attend if instructors offered extra credit. In the faculty development office, we knew that instructors of large enrollment courses would only offer extra credit if it did not add significant work. Using google forms, a mail merge add-on, card swipe readers, and course rosters, we developed a system for students to pre-register, receive reminder emails, and swipe their id cards after the workshop. With this system in place, instructors for over 30 courses received a list of student attendees within a week of the workshop.

To nudge students who attended the workshop to start the Challenge and complete all four weeks, we used two techniques. First, students were told that completing all four weeks would make them eligible for a drawing for ten $100 gift cards to the campus book store. Second, we started the Challenge at the end of the workshop with students selecting one or more strategies to try on a Google form at the end of the workshop.

photo of Ace Your Course Challenge winners
Four of the ten winners of a raffle for students who completed the Ace Your Course Challenge.

The next spiral outwards involved engaging more faculty in a conversation on the powerful ways they could help their students learn. Prior initiatives that had promoted Dweck’s (2016) insights on growth mindsets had primed many faculty and staff for these conversations. Specifically, they wanted to know what else they could do beyond promoting a growth mindset, and a metacognitive approach to learning strategies offered them concrete answers.

In addition, faculty members who had moved away from fixed mindsets about who could succeed in their courses shared their insights on how to approach their still skeptical peers. We developed a strategy of presenting quantitative data alongside student voices to describe the student experience (examples are available here: https://web.uri.edu/atl/ace-your-course-challenge/). Initially, our quantitative data was limited to student self-reports. With the benefit of a snowstorm, we had the chance to organize an IRB approved research project to answer important questions that skeptics raised.

As we shared this data on campus, we were asked to try different permutations of the Metacognition Workshop plus AYC Challenge in two different settings – a support program for conditionally readmitted students and two gateway chemistry courses. In addition, we were asked to offer workshops for professional staff and faculty so they could include McGuire’s approach in their programs and courses.

One of the most successful workshops, “Teach Your Students How to Learn in 50-minutes” provided an annotated version of Dr. McGuire’s slides with breakout discussions about the key messages to motivate students. This led many instructors to experiment with including different elements of her metacognitive approach to learning strategies into their courses.

Some Lessons Learned and Suspected

Each AYC Challenge has generated new data and insights into the potential for URI students to make significant gains in their metacognition. This new data has generated new conversations, which have led to variations on the McGuire workshop and/or the AYC Challenge. This has been a fruitful if unintended process.

Our skeptical internal voice continues to ask how we could nudge more students into participating. We noticed lower participation rates for students from historically marginalized groups in our gateway science course study. This led us to experiment with embedding the workshop plus challenge into courses, but our early experience raised many concerns around overloading instructors and maintaining fidelity with the core AYC challenge experience.

In a promising next iteration, my URI colleague Michelle Fontes-Barros has suggested a partnership with student organizations and clubs, particularly STEM affinity groups for students from historically marginalized groups. Convinced of the value, a student group might sponsor a workshop in a regular meeting space. Student leaders might promote peer commitments to complete the AYC Challenge. Past AYC Challenge participants might help present the workshop and send messages during the Challenge to encourage persistence. This next iteration has the potential to be much more student-centered, but it will be important to critically evaluate the student experience and share results with the wider university community to energize the campus conversation on metacognitive development.

Doyle, T., & Zakrajsek, T. (2013). The New science of learning: How to learn in harmony with your brain. Sterling, VA: Stylus Publishing.

Dweck, C. S. (2016). Mindset: The new psychology of success (Updated Edition). New York, NY: Ballantine Books.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34(10), 906–911.

Kaldor, E., & Swanson, H. (2018, November). A campus-wide strategy to develop metacognition in gateway science courses. Paper presented at the POD Network Conference, Portland, Oregon.

Kaldor, E., & Swanson, H. (2019). How can you elevate metacognition on your campus? Try the Ace-Your-Course Challenge. The National Teaching & Learning Forum, 28(2), 5–7.

McGuire, S. Y. (2015). Teach students how to learn: Strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Sterling, VA: Stylus Publishing.

Nilson, L. B. (2013). Creating self-regulated learners: Strategies to strengthen students’ self-awareness and learning skills. Sterling, VA: Stylus Publishing.