How to Use Post-Test Analysis to Facilitate Metacognition in the College Classroom

by Gina Burkart, Ed.D., Learning Specialist, Clarke University

Pedagogy for Embedding Strategies into Classes

The transition to college is difficult. Students quickly discover that their old strategies from high school do not serve them well in college when they fail their first exam. As the Learning Specialist, I guide these students in modifying strategies and behaviors and in finding new strategies. This also involves helping them move away from a fixed mindset where they believe some students are just born smarter than others and move toward a growth mindset where they reflect on habits and strategies and how to set goals and make changes to achieve desired outcomes. Reflective metacognitive discussion and exercises that develop a growth mindset are necessary for this type of triaging with students (Dweck, 2006; Masters, 2013; Efklides, 2008; VanZile-Tamsen & Livingston, 1999; Livingston, 2003).

As the Learning Specialist at the University, I work with students who are struggling, and I also work with professors in developing better teaching strategies to reach students. When learning is breaking down, I have found that oftentimes the most efficient and effective method of helping students find better strategies is to collaborate with the professor and facilitate strategy workshops in the classroom tailored to the course curriculum. This allows me to work with several students in a short amount of time—while also supporting the professor by demonstrating teaching strategies he or she might integrate into future classes.

An example of a workshop that works well when learning is breaking down in the classroom is the the post-test analysis workshop. The post-test analysis workshop (see activity details below) often works well in classes after the first exam. Since most students are stressed about their test results, the metacognitive workshop de-escalates anxiety by guiding students in strategic reflection of the exam. The reflection demonstrates how to analyze the results of the exam so that they can form new habits and behaviors in attempt to learn and perform better on the next exam. The corrected exam is an effective tool for fostering metacognition because it shows the students where errors have occurred in their cognitive processing (Efklides, 2008). The activity also increases self-awareness, imperative to metacognition, as it helps students connect past actions with future goals (Vogeley, Jurthen, Falkai, & Maier, 1999). This is an important step in helping students take control of their own learning and increasing motivation (Linvingston & VanZile Tamsen, 1999; Palmer & Goetz, 1988; Pintrich & DeGroot, 1990).

Post-Test Analysis Activity

When facilitating this activity, I begin by having the professor hand back the exams. I then take the students through a serious of prompts that engage them in metacognitive analysis of their performance on the exams. Since metacognitive experiences also require an awareness of feeling (Efklides, 2008), it works well to have students begin by recalling how they felt after the exam:

How did you feel?
How did you think you did?
Were your feelings and predictions accurate?

The post-test analysis then prompts the students to connect their feelings with how they prepared for the exam:

What strategies did you use to study?
- Bloom’s Taxonomy—predicting and writing test questions from book and notes
- Group study
- Individual study
- Concept cards
- Study guides
- Created concept maps of the chapters
- Synthesized notes
- Other methods?

Students are given 1-3 minutes to reflect in journal writing upon those questions. They are then prompted to analyze where the test questions came from (book, notes, power point, lab, supplemental essay, online materials, etc.) It may be helpful to have students work collaboratively for this.

An Analysis of the Test—Where the Information Came From

For each question identify where the test question came from:
- Book (B)
- In-class notes (C)
- Online materials (O)
- Supplemental readings (S)
- Not sure (?)

After identifying where the test information came from, students are then prompted to reflect in journal writing upon the questions they missed and how they might study differently based upon the questions they missed and where the questions came from. For example, a student may realize that he or she missed all of the questions that came from the book. That student may then make a goal to synthesizing class notes right after class with material from the book 30 minutes after class, and then use note reduction to create a concept map to study for the next test.

Another student might realize that he or she missed questions because of test-taking errors. For example, she didn’t carefully read the entire question and then chose the wrong response. To resolve this issue, she decided she would underline question words on the test and in attempt to slow down while reading test questions. She also realized that she changed several responses that she had correct. She will resist the urge to overthink her choices and change responses on the next test.

Next, students are taught about Bloom’s Taxonomy and how it is used by professors to write exams. In small groups, students then use Bloom’s Taxonomy to identify question types. This will take about 20-30 minutes—depending upon the length of the test. For example, students would identify the following test question as a comprehension-level question: Which of the following best describes positive reinforcement? Whereas, the following question would be noted as an application-level question: Amy’s parents give her a lollipop every time she successfully uses the toilet. What type of reinforcement is this?

Question Type: Identify What Level of Bloom’s Taxonomy the Test Question is Assessing

Knowledge-level questions
Comprehension
Application
Analysis
Synthesis
Evaluation

Students sometimes struggle with distinguishing the different levels of questions. So, it is helpful to also ask small groups to share their identified questions with the large group, as well as how they determined it to be that level of question. The professor also is a helpful resource in this discussion.

After discussion of the questions types, students then return to individual reflection, as they are asked to count the number of questions they missed for each level of Bloom’s Taxonomy. They are also asked to reflect upon what new strategies they will use to study based on this new awareness.

Adding It All Up

Count the number of questions missed in each level of Bloom’s Taxonomy.
Which types of questions did you miss most often?
Compare this with your study methods.
What adjustments might you make in your studying and learning of class material based on this information? Which levels of Bloom’s Taxonomy do you need to focus more on with your studying?

Finally, students are asked to use the class reflections and post-test assessment to create a new learning plan for the course. (See the learning plan in my previous post, Facilitating Metacognition in the Classroom: Teaching to the Needs of Your Students). Creating the Learning Plan could be a graded assignment that students are asked to do outside of class and then turn in. Students could also be referred to the Academic Resource Center on campus for additional support in formulating the Learning Plan. Additionally, a similar post-test assessment could be assigned outside of class for subsequent exams and be assigned a point value. This would allow for ongoing metacognitive reflection and self-regulated learning.

This type Cognitive Strategy Instruction (Scheid, 1993) embedded into the classroom offers students a chance to become more aware of their own cognitive processes, strategies for improving learning, and the practice of using cognitive and metacognitive processes in assessing their success (Livingston, 2003). Importantly, these types of reflective assignments move students away from a fixed mindset to a growth mindset (Dweck, 2006). As Masters (2013) pointed out “Assessment information of this kind provides starting points for teaching and learning.” Additionally, because post-test assessment offers students greater self-efficacy, control of their own learning, purpose, and an emphasis on the learning rather than the test score, it also positively affects motivation (VanZile-Tamsen & Livingston, 1999).

References

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Balantine Books.

Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13 (4), 277-287. Retrieved from https://www.researchgate.net/publication/232452693_Metacognition_Defining_Its_Facets_ad_Levels_of_Functioning_in_Relation_to_Self-Regulation_and_Co-regulation

Livingston, J. A. (2003). Metacognition: An overview. Retrieved from https://files.eric.ed.gov/fulltext/ED474273.pdf

Masters, G. N. (2013). Towards a growth mindset assessment. Retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?article=1017&context=ar_misc

Palmer, D. J., & Goetz, E. T. (1988). Selection and use of study strategies: The role of studier’s beliefs about self and strategies. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and evaluation (pp. 41-61). San Diego, CA: Academic.

Pintrich, P. R., & DeGroot, E. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40

Pintrich, P. R., & DeGroot, E. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.

VanZile-Tamsen, C. & Livingston, Jennifer. J. A. (1999). The differential impact of motivation on the self regulated strategy use of high- and low-achieving college student. Journal of College Student Develompment, (40)1, 54-60. Retrieved from https://www.researchgate.net/publication/232503812_The_differential_impact_of_motivation_on_the_self-regulated_strategy_use_of_high-_and_low-achieving_college_students

Vogeley, K., Kurthen, M., Falkai, P., & Maier, W. (1999). Essential functions of the human self model are implemented in the prefrontal cortex. Consciousness and Cognition, 8, 343-363.

To Infinity and Beyond: Metacognition Outside the Classroom

by Kyle E. Conlon, Ph.D., Stephen F. Austin State University

My wife, Lauren, and I met in graduate school while pursuing our doctoral degrees in social psychology. Since then, we’ve taught abroad in London, moved to two different states, landed jobs at the same institution—our offices are literally right next to each other’s—bought a house, and had a child. It’s fair to say that our personal and professional lives interweave. One of the great joys of having an academic partner is having someone with whom I can share the challenges and triumphs of teaching. Although we have long promoted the benefits of metacognition in our classrooms, we use metacognition in so many other domains of our lives as well. But the link between metacognitive practice in the classroom and real-world problem solving isn’t always clear for students.

In this post, I’ll discuss how facilitating metacognition among your students can benefit them long after they’ve finished your class, with an emphasis on two important life goals: financial planning and healthy eating.

Metacognition and Money

At first glance, a college student may find little connection between thinking about his or her test performance in an introductory psychology class and building a well-diversified investment portfolio years later. But the two are more intimately linked than they appear. Students who possess high metacognitive awareness are able to identify, assess, and reflect on the effectiveness of their study strategies. This process requires the development and cultivation of accurate self-assessment and self-monitoring skills (Dunlosky & Metcalfe, 2009). As teachers, then, we serve as primary stakeholders in our students’ metacognitive development.

Just as successful students think about their own thinking, successful investors spend a lot of time thinking about how to manage their money—how to invest it (stocks, bonds, REITs, etc.), how long to invest it, how to reallocate earnings over time, and so on. Smart investing is virtually impossible without metacognition: it requires you to continually assess and reassess your financial strategies as the markets move and shake.

Even if your students don’t plan on being the next Warren Buffet, financial thinking will play a central role in their lives. Budgeting, buying a house or a car, saving for retirement, paying off debt—all of these actions require some level of financial literacy (not to mention self-control). Of course, I’m not saying that students need a degree in finance to accomplish these goals, just that they are more easily attainable with strong metacognitive skills.

Indeed, financial security is elusive for many; for instance, the 2018 Report on the Economic Well-Being of U.S. Households found that many adults would struggle with a modest unexpected expense. There are real financial obstacles that families face, for sure. Because financial literacy has broad implications, from participation in the stock market (Van Rooj et al., 2011) to retirement planning (Lusardi & Mitchell, 2007), the transfer of metacognitive skills from academic to financial decisions may be especially paramount.

Admittedly, when I was an 18-year-old college student, I didn’t think much about this stuff. (I was too busy studying for my psychology exams!) But now, years later, living on a family budget, I have a deep appreciation for how the metacognitive awareness I cultivated as a student prepared me to think about and plan for my financial future. For your students, the exams will end, but the challenges of adulthood lie ahead. Successfully navigating many of these challenges will require your students to be metacognitive about money.

Metacognition and Food

As with planning for one’s financial future, eating healthy food is a considerable challenge that involves tradeoffs: Do I eat the salad so I can keep my cholesterol low, or do I enjoy this piece of delicious fried chicken right now, cholesterol be damned? Anyone who’s ever struggled with eating healthy food knows that peak motivation tends to occur shortly after committing to the goal. You go to the grocery store and buy all the fruits and vegetables to replace the unhealthy food in your fridge, only to throw away most of it later that same week. Why is eating healthfully so difficult?

There is an important role for metacognition here. When I teach my Health Psychology students about healthy eating, I draw the habit cycle on the whiteboard: cue à routine à reward (Duhigg, 2012). I tell students that breaking a bad habit requires changing one piece of the cycle (routine). Keep the cue (“I’m hungry”) and the reward (“I feel good”) the same, just change the routine from mindlessly eating a bag of potato chips to purposefully eating an apple. Implicit in this notion is the need to be aware of what you’re eating and the benefits of doing so—in other words, metacognition. Another idea is to have students draw out their steps through the grocery store so they can see which aisles they tend to avoid and which aisles they tend to visit (the ones with processed food). Students gain metacognitive awareness by literally retracing their steps.

In college, I survived on sugar, sugar, and more sugar. (One category short of Buddy the Elf’s four main food groups.) Since then, my metabolism has slowed considerably. Fortunately, with the help of metacognition, I’ve changed my diet for the better. I also cook most meals for our family, so I’m constantly thinking about meal plans, combinations of healthy ingredients, and so on. For me, as for many people, healthy eating didn’t occur overnight; it was a long process of habit change aided by awareness and reflection of the food I was consuming. The good news for your students is that they have several opportunities every day to think intently about their food choices.

The Broad Reach of Metacognition

As a teacher, I love those “lightbulb” moments when a student makes a connection that was previously unnoticed. In this post, I’ve tried to connect metacognition in the classroom to two important life domains. By fostering metacognition, you’re indirectly and perhaps unknowingly teaching your students how to make sound decisions about their finances and eating habits—and probably hundreds of other important life decisions. Metacognition is not limited to exam grades and paper rubrics; it’s not confined to our classrooms. It’s one of those special, omnipresent skills that will help students flourish in ways they’ll never see coming.

References

Board of Governors of the Federal Reserve System (2019). Report on the economic well-being of U.S. households in 2018. https://www.federalreserve.gov/publications/files/2018-report-economic-well-being-us-households-201905.pdf

Duhigg, C. (2012). The power of habit: Why we do what we do in life and business. Random House.

Dunlosky, J., & Metcalfe, J. (2009). Metacognition. Sage Publications, Inc.

Lusardi, A., & Mitchell, O. S. (2007). Financial literacy and retirement preparedness: Evidence and implications for financial education. Business Economics, 42(1), 35‒44.

Van Rooj, M., Lusardi, A., & Alessie, R. (2011). Financial literacy and stock market participation. Journal of Financial Economics, 101(2), 449‒472.

Teaching Study Skills Through Retrieval Practice, Interleaving, and Spaced Practice

by Dana Melone, Cedar Rapids Kennedy High School

We all want our students to study effectively. Many of our students fall into the trap of doing what they have always done. They do this for a lot of reasons. Maybe it has worked for them in the past and now suddenly their method is not working, maybe they were never taught how to study and assumed it must be intuitive, or maybe they have never needed to study in the past. The list can go on and on. As educators, we know that study habits are an important part of academic achievement.

I teach five sections of Advanced Placement Psychology with an average of about 150 students per school year. These students are sophomores, juniors, and seniors from a variety of different socioeconomic, racial and ethnic groups. My students also come in with a variety of experience in taking advanced courses and a variety of ranges of grade point averages. I talk with my students at the beginning of the school year about their study habits to determine whether I need to teach them study skills.

Here are just a few of the patterns of studying/learning misconceptions that I see:

Misconceptions

More time studying means better knowledge of the topic
Re-reading the content or re-reading the notes is effective studying
Re-examining course slides is effective studying
Flashcards of vocabulary will lead to exam success
Listening to music or watching TV while studying is ok
Using books and notes while studying is taking place is enough
Studying looks the same no matter the course or style of exam
Every item in the course must be studied

After seeing these patterns year after year, I wanted to teach my students effective study methods and the use of metacognition in studying. I wanted the focus to be on continuous modeling of this in the course and honest discussions about how this can be used at home as a study tool. For the past few years I have been focusing on clearing up misconceptions about studying, and teaching students methods of retrieval practice, interleaving multiple areas of content, and spacing out their retrieval practice over time for better retention. This focus helps my students provide their own feedback on what they know and do not know.

Retrieval practice is defined as “A learning strategy where we focus on getting information out” (retrievalpractice.org). This means I am teaching my students to talk, write, and answer questions about the content without looking at the material while they are doing it. By focusing on on retrieval over time (spacing of learning), plus weaving in multiple topics from the course at a time (interleaving), I am helping my students become self sufficient in determining what they know and what they do not know about the content and its connection to other units in the course.

I incorporate these three high efficacy learning strategies in my classroom in multiple ways and encourage students to use similar techniques in their own studying. Here are a few examples:

At the end of every unit I provide my students with an online questionnaire that sends them their results and results to me as well. This questionnaire poses questions asking them to state as much information about a current topic as they possibly can, to then connect that information to content from each of our previous units. They then reflect on what they remembered and how well they could connect it to other content areas. They are supposed to answer the questionnaire without referring to notes or text materials.
I provide students with multiple questions from a chapter and have them answer them in as much detail as they can without looking at their notes. They then change ink color and use books and notes answer what they could not originally answer on their own. This exercise provides them with a great visual of what they did and did not know for an entire unit. I call this a brain dump. This name is based off others I have seen in my professional networks. These can be used at any time in a unit, but I prefer to give these to students a week or so prior to an exam.
Another activity similar to a brain dump is a single topic retrieval activity. I take a topic that is addressed over multiple content areas and have them write down as much content as they can without their books or notes by applying it to each area it has appeared in. They then switch ink colors and walk around the room to fill in their gaps. This often includes a prediction of how it might appear in future topics. I call this a retrieval practice challenge. This activity reminds students that we have discussed this topic before from a different viewpoint or lense and allows them to have some predictive discussion of a new lens.

My students have really responded to these methods but it has taken a lot of teaching and reteaching the method and the reasoning behind the use of the method. Many of them have started creating them for their other classes. I have seen a decrease in students asking me for fill in the blank study guides and an increase in students asking me for tips on how to use retrieval practice to study a variety of topics. My plan moving forward is to show students correlation data from their exams showing the relationship between retrieval practice and test scores.

Being Authentic: Modeling Metacognitive Growth and Connecting with Students

by Patrick Cunningham, Ph.D., Rose-Hulman Institute of Technology

Learning takes effort and can feel hard at times. Likewise, learning to manage one’s learning processes better, growing metacognitively, is an effortful process. Because approaches to learning are habitual, deeply ingrained over time, changing them is hard. Metacognitive growth often progresses in fits and starts with cycles improvement and relapse into old habits. This is true for our students and for us as instructors. Remembering this can help us help them better with metacognitive lessons that can guide them throughout their lives.

I often say that teaching students about metacognition and engaging them in their metacognitive growth is one of the most important and authentic things I do academically. I have expertise within my field (Mechanical Engineering, Dynamic Systems and Control) and I am accomplished at applying it to engineering problems. When I engage students in learning this content, I model this expertise. I strive to make my content-focused teaching authentic, but it isn’t authentic in the same way as providing instructional experiences for my students on metacognition. While I may know which disciplinary concepts students will likely struggle with, my struggles with those concepts are a distant memory. However, as I engage my students in metacognitive growth, i.e., changing my habitual approaches to learning, my struggles are concurrent with theirs.

Becoming a student again

For example, I have encountered my metacognitive struggles as I have been learning German, initially as a hobby and now more intentionally as I prepare for a sabbatical in Germany. About a year-and-a-half ago I decided to sign up for a Duolingo account to see if I could resurrect and build on my prior German language experience. I had just signed my children up for accounts to practice Spanish and sensed an opportunity to finally work towards a long-held personal goal – becoming proficient in another language. Armed with my knowledge of metacognition, I wanted to make my practice effective and efficient. I set a modest but good practice goal of two grammar modules per day, and I jumped in. I have been consistent in my daily practice – only missing about three days in 18 months – and I have added some varied strategies to practice German – Duolingo stories (spoken dialogues with text), trying to use basic phrases in my daily life, and trying to read German news stories. However, I have also noticed some metacognitive pitfalls in my language practice, even as I have gotten more serious about it.

I have not engaged in recall practice, despite the pop-up messages reminding me that I should take time after a session to recall new vocabulary and key grammar and usage insights. I also know its value, but I tell myself I just don’t have time. I am avoiding a good strategy and making poor use of my metacognitive knowledge of strategies and tasks.

I also lack a clear learning goal. What does proficient mean? How will I know I have achieved it? What are appropriate incremental goals that build towards proficiency? Admittedly, for most of this time, this has been more of a hobby pursuit, but if I really want to develop the skill, I need more specific, even if still modest, learning goals. This is poor metacognitive planning.

Then there is how I track my progress. I am sad to say I have taken pride in the number of XP points (virtual points within Duolingo) I have accrued and the number of modules I have crossed off. They are easy to count, but this does not really assess my proficiency with the German language accurately. This is poor metacognitive monitoring of my learning.

Within the story modules, I am tentative and find myself relying too much on being able to hover over the words to see the definition before answering the comprehension questions. Why do I find it hard to commit to an answer, right or wrong, and learn from it? It feels hard, but as I tell my students, this is how you know you are learning. I am relying on a lower quality strategy because it feels good.

Acknowledging the Same Struggles

So, each time I bring up metacognition with my students, I am faced with the reality of my struggles with it, as demonstrated by the pitfalls in my German language practice. Thus, I teach about metacognition, not as an expert who has it all figured out, but rather as one who is, perhaps, further along the path. This is humbling. So, what can I do with this struggle? Can I really engage my students in their metacognitive growth if I am struggling with my own metacognitive growth? Yes! But how I go about it matters.

If I ignore talking about metacognition altogether, then I might avoid feeling uncomfortable about my failings with it. But I would not likely grow myself nor help my students grow metacognitively. If I pretend I have it all figured out, then I risk being found-out, losing my credibility, and sabotaging my students’ potential metacognitive growth.

However, if embrace my struggles with metacognition and am honest about them with my students, then I might be able to grow myself while I also help my students. There are at least two mechanisms for positive impact, acknowledging my similarity to my students and providing my students a concrete model of metacognitive growth in practice.

When I accept that I am like my students and my students are like me in struggling with metacognitive practice and growth (e.g., my German language studies), I gain a more complete view of my students. I can no longer view them as just lazy, unmotivated, or lacking in work ethic. Instead I can see sincere effort and a desire to learn and do well in classes, in spite of less effective learning behaviors. When I see my students in this way, I have a better attitude when I interact with them. This enables me to authentically praise the positive aspects of their learning behavior and to more gently challenge the less effective aspects they are relying on. It can move our interaction from a place of discouragement to one of encouragement and can help students to view their ability to grow into the learning challenge before them. When I recognize that my students have similar barriers to metacognitive growth as I do, I am able to be more compassionate and supportive as I help them face the challenges they experience in their learning.

Points of Connection

Sharing my own metacognitive struggles, e.g., with my German language practice, can provide a point of connection with my students. Students can have a hard time identifying with their instructors, viewing us as experts with experiences far removed from theirs. It is heartening to see students warm up to me and talk more openly and honestly about their approaches to learning when I have shared elements of my struggles with learning German. Suddenly, the relational distance between us shrinks because I have a present learning experience, concurrent with theirs, that they can readily identify with. Such authentic connections build trust and a foundation for a relationship, which can lead to further support and processing of their learning experiences.

Beyond forging an authentic connection, I can also constructively model of the effortful and continual path to becoming a more skillful learner, i.e., metacognitive growth. Sharing my critiques of my German language practice can demonstrate metacognitive evaluation of my learning processes and my openness to ongoing refinement. However, metacognitive growth does not stop with recognizing ineffective or less effective learning strategies. It requires doing something about it, enacting a productive change. For example, I could share a more refined and specific goal – such as, wanting to be able to engage in pleasantries and make small talk in German – which also helps direct my practice and how I monitor my progress.

How can this look within a specific class? This winter I have been teaching computer programming, and I have found it useful with a few students to draw analogies between my language practice and working towards proficiency with programming concepts. Becoming conversant in German requires more than just knowing German grammar rules, e.g., declensions for accusative cases. I must practice using it in conversation, that is, applying it. Similarly, to become proficient with programming I must know how to write conditional statements and loops, but I also must know how to apply them in various ways to accomplish a specified task – I have to practice applying the concepts. I was trying to model the need to align learning strategies with specific learning goals.

In summary, if I can identify with my students, I can better help them with their metacognitive growth. It helps me to be more gentle and supportive in my desire to see them grow metacognitively. It can also can help my students connect with me and see that they too can persist in their metacognitive growth when I am forthright with my metacognitive shortcomings. Being authentic matters and it can help me do what I think is the most important teaching that I do, helping my students become better learners. These are enduring lessons that can help them be successful throughout their lives, even if they forget the content that I was teaching!

Facilitating Metacognition in the Classroom: Teaching to the Needs of Your Students

by Gina Burkart, EdD, Learning Specialist, Clarke University

Once instructors understand the strengths and weaknesses of students, they can begin to adapt how they deliver the content of their course in ways that their students will be able to connect with it. This makes self-assessments valuable tools for learning and teaching (Burkart, 2020). Self-Assessments help professors note students’ strengths, weaknesses, and perceptions of self so that they can teach curriculum to the needs of the students. This allows professors to use metacognition in their own teaching, i.e. metacognitive instruction.

Mapping Out a Plan for Teaching and Learning

In my own courses, student assessments guide me in using a chart to map out a plan of how to address students’ needs while teaching the planned curriculum on the syllabus. This allows me to be mindful of my teaching and how it is connecting with students as the semester unfolds. The chart helps me note and monitor my students’ strengths and weaknesses, how these may impact course goals/outcomes and strategies that I will use to help my students adapt to the content demands of the course. See Figure 1 for an example chart.

Likewise, it is also helpful to guide students in using metacognition to create their own learning plans for the semester. Creating learning plans can be an effective activity for the first meeting of the class, while going through the course syllabus.

To facilitate this activity, a professor might hand out the syllabus, give students 5-10 minutes to read through the syllabus, and then have them share responses to the following questions in small groups to more strategically and critically read the syllabus and discuss it with classmates:

What prior reading and writing experiences have shaped you as a learner?
Based on this course syllabus, what challenges do you anticipate?
What strategies and resources will you use to meet these challenges?
What are your goals for this course?
How can you connect the material of this course with your major?
How can we as your learning community support you with your learning and goals? What do you need from us?

The small groups can then be invited to share their responses with the large group and write them on the board. Professors might then take a picture of the board and refer back to it later to use as an informal student assessment to inform teaching.

Finally, students return to small groups to create a more detailed and personalized plan of what they might do in order to find success and meet the outcomes of class. Professors might provide a template of a learning plan that would guide students in going through the syllabus and pulling out key information to create the plan. See Figure 2 for an example student learning plan.

Having these types of activities and discussions at the beginning of a course empowers students, as they realize that they are in charge of their learning. As Biggs and Tang (2011) have recognized, student engagement increases when students feel co-ownership and empowerment in the classroom.

Adaptation for Online Courses

This metacognitive activity can also be adapted in an online course through Voice Thread. Voice Thread allows students to create and upload an interactive, video recording of themselves. I have asked students to respond to the same questions in a 3-5-minute Voice Thread video and to also respond to a classmate’s Voice Thread. Additionally, after completing the Voice Threads, students were required to electronically submit a learning plan based on the syllabus for course credit. Similar to the in-class discussions in the face-to-face class, this activity fosters the metacognitive monitoring process that Flavell (1979) described. As students reflect on the skills necessary for the course outcomes, prior metacognitive knowledge and experiences resurface as goals are set and strategies for achieving the goals are identified and shared. Professors are then also able to monitor the needs of the students and provide feedback related to goals and strategies.

In both instances, students are able to share about their learning in a social setting, which reminds students they are not alone in their learning. This is important because students learn about their own learning and self by reflecting on the learning of others (Mead, 1962/1934; Burkart, 2010).

Assignments and Activities that Continue to Foster Metacognitive Growth throughout the Semester

Professors can continue to foster metacognition throughout the semester by integrating assignments and activities that reinforce reflection on learning, strategies, and goal setting. For example, professors might begin each class period with a 1-minute pre-write where students list main points from the assigned readings, questions they want to discuss in class, and why the material is relevant and important. I often have students do this on an index card, and I collect them and use the cards to guide my teaching during the class. While students are engaged in group work, I quickly go through the cards, award a check, plus, or minus with brief comments (to show I value metacognitive work) and incorporate these into class participation points. Questions that students wanted to discuss in class are then discussed. Sometimes, I pose the questions to the large group for discussion. Other times, I distribute the questions to small groups and assign them to formulate a response for the rest of the class.

Not only does using students’ questions in class show students that they are being listened to, it acknowledges that their questions are valued and their preparatory work outside of class is connected to learning inside the classroom. And, based on their questions and class discussion, I rethink strategies and interventions to help students better access the course materials. As Simmons (2017) noted, this type of participatory pedagogy makes students more aware of their own cognitive processes.

The following is a list of activities that I have used or recommended to faculty to foster student metacognition in the classroom (Burkart, 2019):

1-minute prewrite at the beginning of class
Create or model note-taking and reading that facilitates learning
Model thinking aloud with problem solving, reading, interpreting
Incorporate learning checks
Assign post-test or assignment analysis
Encourage learning material on all levels of learning by assessing and modeling strategies scholars use: concept mapping, reading journals, study/note journals, portfolios
Reward revision, rethinking, growth by awarding points to final products
Use wrappers while teaching
Encourage or arrange study groups
Relate the material to students’ lives and experiences
Create and assign course blogs with prompts that encourage reflection and discussion on goals, use of strategies, and challenges and growth with learning course material
Create group tests/assessments

While monitoring students’ needs and growth throughout the semester, professors can use this list to use metacognition in their own teaching—establish goals and integrate strategies to impact student learning. In turn, this will guide students in reflecting on their learning and increase engagement. Most importantly it provides practice in metacognition that empowers students to take control of their own learning that will carry over into other classes and their personal lives outside of the classroom.

References

Biggs, J. B., & Tang, C. (2011). Teaching for quality learning at the university: What the student does. Maidenhead, UK: Society for Research into Higher Education & Open University Press.

Burkart, G. (2019). Engaging the unengaged in the college classroom. Faculty workshop—by request, Clarke University, Dubuque, IA.

Burkart, G. (2010, Dec). First-Year College Student Beliefs about Writing Embedded in Online Discourse: An Analysis and Its Implications to Literacy Learning. (Unpublished doctoral dissertation). University of Northern Iowa, Cedar Falls, IA.

Burkart, G. (2010, May). An analysis of online discourse and its application to literacy learning, The Journal of Literacy and Learning. Retrieved from http://www.literacyandtechnology.org/uploads/1/3/6/8/136889/jlt_v11_1.pdf#page=64

Mead, G. H. (1934). Mind, self, and society: From the standpoint of a social behaviorist. Chicago: University of Chicago Press.

Simmons, N. (2017). Participatory Pedagogy: Inviting Student Metacognition. Retrieved from https://www.improvewithmetacognition.com/developing_student_metacognition_simmons/

Getting to Know Your Students: Using Self-Assessment in the Classroom to Foster Metacognition

by Gina Burkart, EdD, Learning Specialist, Clarke University

The Process of Metacognition

As retention continues to dominate discussions at most universities, metacognition may provide much insight. Flavell’s (1979) early and hallmark work on metacognition defined metacognition as an individual’s reflection on how he or she learns and developed a model to depict the process of this reflection. According to Flavell (1979), metacognition “occurs through the actions and interactions” of “metacognitive knowledge,” “metacognitive experiences,” “goals/tasks,” “actions/strategies” (p. 906). Understanding how the metacognitive process impacts learning is key to developing effective curriculum, helping students learn material, and motivating students to learn.

Self-Assessment to Enhance One-on-one Mentoring

As the Learning Specialist at Clarke University, one of my responsibilities is meeting with, monitoring, and guiding students in finding effective learning strategies. In this role, I meet with students one-on-one, reach out to students who have received student concerns flags raised by professors, create and coordinate academic support (Academic Coaching and Supplemental instruction), collaborate and guide faculty in developing curriculum through workshops and consultations, and hire, train and supervise the Academic Coaches in the Academic Learning, mentor and meet with students placed on Academic Warning and Probation, and teach the College Study Strategy course and courses in the English department.

In working with students who have been placed on probation and warning, I find that students often fail because they lack motivation and purpose. And, commonly, the motivation and purpose have been affected by inaccurate metacognitive knowledge. Flavell’s (1979) model of Cognitive Monitoring offers a schema for understanding how this might occur and how to help students find motivation and purpose and improve their academic standing.

As noted earlier, Flavell (1979) found that our metacognitive knowledge is informed by our metacognitive experiences. Thus, negative experiences or experiences where distorted thought processes created inaccurate metacognitive knowledge about self might result in a lack of purpose or motivation. For example, if a first-year student fails two tests in Biology and compares himself or herself to some classmates who received As, he or she might conclude that he or she is incapable of learning Biology, is not capable of ever becoming a doctor, and should not attend college.

In meeting with the student, I would help the student reflect on how he or she was reading, studying, and taking notes in the Biology course. Additionally, I would help the student reflect on time management and organization strategies. I would also point out the flaw in comparing oneself with others in assessing one’s own abilities. Once the student realizes the flaws of thinking and forms new metacognitive knowledge and experiences, he or she works with me to establish realistic goals and implement new strategies for achieving the goals. Motivation and purpose then quickly improve, and students find success. In some instances students have moved from academic probation to Dean’s List in as little as one semester.

Helping students find success involves helping them discover what they believe about themselves (metacognitive self knowledge), setting goals and finding strategies to achieve the goals. To begin this process, students must first reflect on and assess themselves. As research has shown, unless the self-system is activated, learning will not occur (Mead, 1962/1934; Bandura, 1994; Marzano, 2001; Piaget, 1954; Vygotsky, 1934/1987; Burkart, 2010).

Incorporating Self-Assessment in the College Classroom

In addition to working with students in one-on-one mentoring, I have also found that this type of cognitive monitoring can be fostered in the classroom through the use of self-assessments. As demonstrated by Taraban (2019), self-assessments can be simple or more nuanced depending on the preferences of the professor and the needs of the course curriculum. The self-assessment can be created by the professor or be a nationally normed assessment. Additionally, the assessments can be closely connected to the outcomes of the course and revisited throughout the semester.

I have integrated self-assessments into my own teaching in a variety of ways. For example, in the College Study Strategy course that I teach, I begin the semester with an informal self-assessment by having students rate themselves (5 high and 1 low) in the following course content areas that impact academic performance: reading, time management, organization, test taking, and studying. Additionally, I have them identify strengths and weaknesses in each of these areas and set goals (See Figure 1 for a sample self-assessment and Figure 2 for a sample goal-setting chart).

Students then complete a more formal self-assessment, the nationally normed LASSI (Learning and Study Strategies Inventory). This self-assessment is quick and easy (takes about 10 minutes) and allows students to see how they compare nationally with other students taking the inventory in the following areas: Selecting Main Ideas, Information Processing, Time Management, Self-Testing, Motivation, Concentration, Attitude, Use of Academic Resources, and Test Taking. Students then share their assessments with each other in pairs and large group discussions. In almost all cases, the LASSI and informal assessments match and students find the LASSI results to be accurate. The comparison of data from a nationally normed self-assessment with an informal self-assessment offers students a way for checking the accuracy of their knowledge of self.

These assessments also provide purpose and focus for the course. Class discussion based on the self-assessment establishes buy-in from the students as they see personal need for the course. Additionally, I have found that starting the semester with these assessments frames the course in that I (as the professor) have a better understanding of their skill levels and needs and can connect their assessments to the course curriculum and outcomes.

For example, in Week One when we are going over Time Management and time management strategies, I can refer back to the students’ self-assessments and goals. Asking the students to recall their scores and goals begins the process of cognitive monitoring (Flavell, 1979). It creates purpose and motivation for the students to learn the curriculum I am teaching and integrate it into their courses so that they will begin to develop and apply the new time management strategies in order to achieve their time management goals.

Students are then tasked with implementing the strategies in their courses and asked to display artifacts of the implemented strategies in a midterm and final portfolio that is shared in a personal conference with me. For example, a student may include a long-term planner of the semester with mapped out projects, papers, tests, and athletic games to show that they have started to use macro-level planning for time management. They might also include sample pages from a weekly planner to show prioritized “to-do” lists and items crossed off—micro-level planning.

Students also assess themselves again with the same informal self-assessments at midterm and at the end of the semester. Additionally, they retake the LASSI at the end of the semester and use the self-assessments and artifacts to compile a portfolio that includes a one-page reflection. In the final conference meeting with me, students use the portfolio to demonstrate their growth, as they discuss their goals, strategies used, plans for future goals, and growth.

Integrating Self-Assessment—As a Tool of Metacognition

In assessing themselves, students gain knowledge of what they believe about themselves and how they learn. In reflecting on their assessments and discussing their experiences with me in conferences and with other students in the class, students uncover inaccurate perceptions of self. Additionally, they form goals and learn and develop strategies that positively affect their college learning experience. This sharing of information also allows me, as the professor and Learning Specialist, to also engage in metacognition as I teach and develop curriculum to meet the needs of my students throughout the semester (Burkart, 2017). And while some may question the validity of self-assessments, Nuhfer (2018), found self-assessments to not only be valid but also to be useful tools for both professors and students to monitor learning.

Above I offered examples of how self-assessment is easily integrated into a college study strategy course; however, it can easily be integrated into any course. For example, in teaching literature or writing courses, I create self-assessments unique to that content area and the course outcomes. In literature courses, on the first day of the semester, I ask students to assess themselves in the following areas: critical reading, writing, speaking, time management, and small group work. I also have students read through the syllabus, create goals for each of those areas, and identify strategies they will use to achieve those goals. Additionally, I have them respond to the following questions:

What do you hope to get out of this course? How does it connect with your career and life goals?
How can I help you achieve your goals?
What challenges do you anticipate this semester? What resources are available to help you meet those challenges?
What else do you want me to know about you and what you have going on this semester?

Students share their assessments in small groups. Then, as a large group, we discuss the assessments and the syllabus. I collect the assessments, comment on them, and then return them. Students refer back to them again at midterm and at the end of the semester when they complete synthesis reflections about their growth and achievement of course outcomes.

Benefits of incorporating student self-assessment

The inclusion of these assessments has been helpful in many ways. They have helped students feel that they are listened to by their professor. The assessments also assist me in quickly and easily conducting a needs assessment of my students so that I can reflect upon and adjust my teaching to their needs (i.e. engage in metacognitive instruction).

Most importantly, it encourages students to reflect on their own learning and empowers them to take control of their learning and results in increased motivation and a sense of purpose; this is the power of metacognition and why it matters to retention. When this recursive process activates the self-system (Mead, 1962/1934; Bandura, 1994; Marzano, 2001; Piaget, 1954; Vygotsky, 1934/1987; Burkart, 2010), it develops grit and a growth mindset (Burkart, 2010; Duckworth, 2019; Dweck, 2007). And as Flavell (1979) noted, fostering cognitive monitoring is an important part of learning, as there is “far too little rather than enough or too much cognitive monitoring in this world” (p. 910).

References

Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Ed.), Encyclopedia of human behavior (Vol. 4, pp. 71-81). New York: Academic Press. Retrieved from http://www.des.emory.edu/mfp/BanEncy.html

Burkart, G. (2017, 3^rd ed). 16 weeks to college success. Kendall Hunt: Dubuque, IA.

Burkart, G. (2017, fall). Using the LASSI to engage metacognitive Strategies that foster a growth mindset in college students placed on academic probation (per request). LASSI in Action. Retrieved from https://www.hhpublishing.com/ap/_assessments/LASSI-in-Action-Articles/LASSI-In-Action-Fall-2017.pdf

Duckworth, A. L., Quirk, A. Gallop, R., Hoyle, R. H., Kelly, D. R., & Matthews, M. D. (2019). Cognitive and noncognitive predictors of success. Proceedings of the National Academy of Sciences, 116(47), 23499-23504. Doi:10.1073/pnas.1910510116.

Dweck, C. (2007). Mindset: The new psychology of success. New York: Ballantine Books.

Flavell, J. (1979). Metacognitive monitoring: A new area of cognitive development inquiry. American Psychologist 34(10), 906-9-11.

Marzano, R. J. (2001). Designing a new taxonomy of educational objectives. Thousand Oaks, CA: Corwin Press.

Mead, G. H. (1934). Mind, self, and society: From the standpoint of a social behaviorist.

Nuhfer, E. (2018). Measuring metacognitive self-assessment: Can it help us assess higher-order thinking. Improve with Metacognition. Retrieved from

Piaget J. (1959). The language and thought of the child. Hove, UK: Psychology Press.

Taraban, R. (2019). The metacognitive reading strategies questionnaire (MRSQ): Cross-cultural comparisons. Improve with Metacognition. Retrieved from https://www.improvewithmetacognition.com/metacognitive-reading-strategies/

Vygotsky L. S. (1934/1987). Thinking and speech. The collected works of Lev Vygotsky (Vol. 1). New York, NY: Plenum Press.

Revising the Metacognitive Awareness Inventory (MAI) to be More User-Friendly

By: Melissa Terlecki, PhD, Cabrini University PA

Background

Measuring metacognition, or the awareness of one’s thoughts, is no easy task. Self-report may be limited and we may overestimate the frequency with which we use that information to self-regulate. However, in my quest to assess metacognition, I found the MAI, or the Metacognitive Awareness Inventory (Schraw and Dennison, 1994). The MAI, comparatively speaking, is one of the most widely used instruments. Schraw and Dennison found an alpha coefficient of .91 on each factor of the MAI and .95 for the entire MAI, which indicates reliability. Pintrich (2000) agrees the MAI has external validity given MAI scores and students’ academic achievement are highly correlated.

The Problem

Despite the wide use and application of the MAI, I found the survey measurement scale unfitting and constrictive. The survey consists of 52 questions with true or false response options. Some of the behaviors and cognitions measured on the MAI include, “I consider several alternatives to a problem before I answer,” “I understand my intellectual strengths and weaknesses,” “I have control over how well I learn,” and “I change strategies when I fail to understand”, just to name a few (see https://services.viu.ca/sites/default/files/metacognitive-awareness-inventory.pdf).

Though these questions are valid, to dichotomously respond to an extreme “true”, as in I always do this, OR a “false”, as in I never do this, is problematic. Yes-No responses also make for difficult quantitative analysis. All or nothing responses makes hypothesis testing (non-parametric testing) challenging. I felt that if the scale was changed to be Likert-type, then participants could more accurately self-report on how often they may exhibit these behaviors or cognitions, and we could more readily assess variability and change.

The Revised MAI

Thus, I revised the MAI to use a five-point Likert-type rating scale, ranging from “I never do this” to “I do this always” (see Figure 1). Five points also allows a middle rating with two extremes on either side (always/never). It is important to note that the original content of the survey questions has not been altered.

My recent findings (Terlecki & McMahon, 2018; Terlecki & Oluwademilade, in preparation) show the revised MAI to be effective as a pre- and post-test measure to assess the growth due to metacognitive instruction, compared to controls with varying levels of instruction, in college students.

Five example MAI items with the new Likert-scale response options.

Figure 1. Revised MAI likert-scale (Terlecki & McMahon, 2018). Response scale adapted from Schraw and Dennison (1994) with permission from Sperling (Dennison).

In our longitudinal sample of roughly 500 students, results showed that students exposed to direct metacognitive instruction (across a one semester term) yielded the greatest improvements on the revised MAI (compared to controls), although maturation (age and level in school) had a moderating effect. Thus, we concluded that students who were deliberately taught metacognitive strategies did exhibit an increase in their cognitive awareness, as measured by the revised MAI, regardless of initial levels of self-awareness. In other words, the older one is, the greater the likelihood one may be self-aware; however, explicit metacognitive intervention still boasts improvements.

These changes might not have been elucidated using the original, dichotomous true/false response options. The revised MAI is a useful tool in measuring such metacognitive behaviors and whether changes in frequency may occur over time or intervention. Likewise, anecdotal evidence from my participants, as well as researchers, supports the ease of reporting using this Likert-scale, in comparison to the frustration of using the 2-point bifurcation. Still, usage of the revised MAI in more studies will be required to validate.

Suggestions for Future Usage of the MAI & Call for Collaboration

The Metacognitive Awareness Inventory (MAI) is a common assessment used to measure metacognition. Quantifying metacognition proves challenging, yet this revised instrument appears promising and has already provided evidence that metacognition can grow over time. The addition of a wider range of response options should be more useful in drilling down to frequency of usage of metacognitive behaviors and thinking.

Validation studies on the revised scoring have yet to be conducted, thus if other researchers and/or authors are interested in piloting the revised MAI, please contact me (* see contact information below). It would be great to collaborate and collect more data using the Likert-form, as well as have a larger sample that would allow us to run more advanced statistics on the reliability and validity of the new scaling.

References

Pintrich, P.R. (2000). Issues in self-regulation theory and research. Journal of Mind and Behavior, 21, 213-220.

Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460-475.

Terlecki, M. & McMahon, A. (2018). A call for metacognitive intervention: Improvements due to curricular programming and training. Journal of Leadership Education, 17(4), doi:10.12806/V17/I4/R8

Terlecki, M. & Oluwademilade, A. (2020). The effects of instruction and maturity on metacognition (in preparation).

*Contact: If looking to collaborate or validate the revised instrument, please contact Melissa Terlecki at mst723@cabrini.edu.

If you’d like to compare the MAI to other metacognition assessment inventories, please see “A Non-Exhaustive List of Quantitative Tools to Assess Metacognition” by Jessica Santangelo.

Metacognitively meeting students where they are

John Draeger, SUNY Buffalo State

The Teaching and Learning Center at SUNY Buffalo State hosts a regular breakfast conversation on Friday mornings. Faculty and staff gather around coffee, bagels, and a common reading. This fall we read Becoming a Student-Ready College: A new culture of leadership for success (McNair et al., 2016). The book chronicles recent changes in higher-education. More and more students are the first in their families to go to college. More come from communities of color. More work full-time and find it necessary to take care of family members while attending school. I love the underlying premise of the book. The question is NOT what students are doing to ready for us. Rather, the question is what WE are doing to get ready for THEM.

This post argues that promoting metacognition is necessary if we are going to meet the needs of our current students. Metacognition prompts us to be aware of what students need and points the way towards the necessary adjustments.

We can do better and we must do better. Students are stopping out at alarming rates. In some cases, this is simply because they didn’t successfully navigate obtuse institutional expectations. For example, how many times do we ask students to crisscross campus for this signature on that form only to be told they need to come back later? Come to think of it, how many institutional procedures actually make sense? Many policies kept alive by sheer institutional inertia. Those of us who have been around long enough may know how to navigate them, but this is profoundly unfair to students.

McNair et al. encourage each of us to ask the questions that pave the way for student success, even when (perhaps especially when) it is uncomfortable. They call on campuses to embrace a leadership model that is simultaneously bottom-up, top-down, and inside-out. Everyone must take it upon themselves to live into their role in creating a study-ready campus.

Senior leadership needs to articulate a vision of the campus needs to do to be ready for students. Shared governance bodies need to hold the campus accountable. Offices across campus can, and should, reflect on the policies, procedures, attitudes, and behaviors that might inadvertently impede student success. While no one person can transform the campus, each person can clean up their own corner of it. How can departments align learning outcomes with the needs of current students? How can faculty explore whether and how their classroom methods facilitate student learning of all students? And how can everyone on campus be mindful of the many points of view represented by our diverse student bodies, and strive to overcome stereotypes and implicit bias to demonstrate a steadfast belief in students?

Each chapter of the book offers a series of guiding questions to frame campus discussions. They are, in essence, offering a metacognitive approach to culture change. It can happen. Jen McCabe and Justine Chasmar outline how Goucher College has been transformed through metacognition (McCabe & Chasmar, 2018). Of note, the Goucher initiative was kicked off by their president, Jose Antonio Bowen. While it takes a campus become student-ready, senior leadership can play an important role in framing the conversation. Regardless, the conversation needs to happen if campuses are to meet students where they are.

This site defines ‘metacognition” as “an intentional focusing of attention on a process in which one is personally engaged. It encourages awareness of one’s current state of accomplishment, along with the situational influences and strategy choices that are currently, or have previously, influence accomplishment of that process.” Becoming student ready requires that each and every person be personally engaged with student success. Individuals and offices across campus need to critically assess the current state of affairs. Is this a student ready campus? If not, then what are the situational influences keeping that from happening?

If change is necessary, then what strategy choices are mostly likely to influence that process? Progress should be monitored so that all involved are aware of the current state of accomplishment. If the current strategies are not resulting in a student-ready campus, then further thought should be put to how campuses will make the necessary adjustments. In short, metacognition is necessary if colleges and universities are to become student-ready.

References

McCabe, J. & Chasmar, J. (posted December 10, 2018). “Metacognition at Goucher I: Framework and Implementation.” Retrieved from https://www.improvewithmetacognition.com/metacognition-at-goucher-i-framework-and-implementation/.

McNair, T. B., Bensimon, E., Cooper, M. A., McDonald, N., & Major Jr, T. (2016). Becoming a student-ready college: A new culture of leadership for student success. John Wiley & Sons.