Month: June 2019

Wrapping up Metacognition: Pre- and Post-Exam Interventions

By Jennifer A. McCabe, Ph.D., Goucher College

Multiple studies have demonstrated that college students report using less-than-optimal learning strategies when preparing for exams. Without explicit instruction on effective techniques, along with guidance on how to engage in metacognitive monitoring and evaluation of their learning processes, it is not clear how this situation will improve. One of the many ways in which this goal could be achieved is through a specific technique called “exam wrappers.”

"Wrap it up" slogan

An exam wrapper (also known as a “cognitive wrapper”; Bowen, 2017) is a brief activity in which students complete a form to assess their recent exam performance, describe and reflect on how they prepared, and make a strategic plan for future improvement. This would typically be given to students upon receiving exam grades, with the goal to shift the focus from course content and exam outcome (grade) to the learning process itself. Since being introduced by Marsha Lovett in 2013, educators have been encouraged to use this tool to improve student metacognition and, ultimately, performance on exams and assignments.

There is surprisingly little well-controlled research on exam wrappers, and the several studies that have evaluated their impact are lacking in statistical power, internal validity, and/or generalizability. Raechel Soicher and Regan Gurung note this issue at the start of their 2017 article in which they report the results of an exam-wrapper intervention in introductory psychology. They compared an exam wrapper (modeled on Lovett, 2013) to both a “sham wrapper” condition in which students evaluate their incorrect answers and connect each to a relevant course topic, and also to a true control condition in which students simply reviewed their exams without explicit instruction. Results showed no differences among conditions in final grades (even when controlling for pre-intervention metacognition scores), nor on any of the exams, nor on metacognition subscale scores. The authors suggest that exam wrappers may be more successful when used across multiple classes, and that it may also help to make them more interesting and engaging for students. As I suggest below, perhaps having students complete the exam wrappers in the context of having learned about effective study strategies would also improve the benefit of implementing them after exams.

Another recent study, published in 2017 by Patricia Chen and colleagues, reported on outcomes from an exam-wrapper-type of activity called a “Strategic Resource Use” (SRU) intervention. Students in an introductory statistics course were randomly assigned to the SRU intervention or to a control condition that experienced many parts of the activity except for the focused metacognitive components. Importantly, this approach differs from that of traditional exam wrappers in that (1) it was self-administered and fully online; and, more importantly (2) there were both pre- and post-exam components. In the 7-10 days prior to taking the exam, all students completed an online survey in which they reported their predicted exam grade, motivation level, importance of achieving that grade, and confidence in reaching their performance goal. Those in the SRU condition also answered questions about the upcoming exam format, the types of resources available to them during preparation time, why each would be useful, and their plan for using each one. From a checklist of class resources, SRU students provided elaborated answers on usefulness and strategic planning. After the exam, students reported on which they had used, level of perceived usefulness, and how much self-reflection they had engaged in with regard to learning course material. Results showed that in comparison to the control condition, SRU students had higher course grades (about 1/3 of a letter grade), lower self-reports of negative affect toward exams, and higher perceived control over exam performance.

It is interesting that Chen and colleagues do not make the connection to the exam wrapper idea or literature. Both interventions described above have similar implementation and goals surrounding exams – to improve undergraduates’ self-regulated learning by focusing their attention on how they currently learn, how the quality and/or quantity of preparation map on to exam performance, and how they can use various strategies to improve for next time. Both interventions are based on the idea that highlighting the essential metacognitive processes of reflecting and adjusting supports student learning.

What to do with this mixed evidence and varying models for implementing this metacognitive “wrapper” tool? I have personally been using post-exam wrappers (modeled on Lovett) in my Cognitive Psychology course for several years. Though I have not collected empirical data on their effectiveness, based on student comments and my own observations I believe they help and plan to continue to use them. After considering Soicher and Gurung’s methods and results, I think that my implementation may be especially poised for single-course success because, unlike in the two studies discussed above, my exam wrappers are administered on the heels of learning about and engaging in practice with evidence-based learning strategies such as elaboration and frequent, effortful, and distributed (spaced) retrieval practice.

In addition to incorporating these elements into my course structure to provide students with multiple tools for durable learning, they also read the book “Make It Stick” (Brown, Roediger, and McDermott, 2014) early in the semester and engage in writing and peer discussion about effective ways to learn as described in my 2017 blog post Make It Stick in Cognitive Psychology. Thus, when my students complete the post-exam wrapper by reporting strategies they used, and those they will try to increase for future exams, they are doing so in a context of this metacognitive knowledge and accompanying motivation to learn. I am planning to add a pre-exam wrapper component, similar to the SRU model, the next time I teach this course, and given Chen et al.’s promising results, I hope it will even further support my students’ metacognitive development, learning, and, yes, course performance.

I explicitly communicate my perspective on exams to students, early and often: tests are learning events. By incorporating exam wrappers, I am reinforcing this message, and my students see that I care about their learning and my genuinely want them to improve. This also connects to a chapter in “Make It Stick” on the benefits of having what Carol Dweck calls a growth mindset – believing that intelligence is malleable and can be enhanced through practice and strategic effort. I encourage my students to adopt this mindset in multiple ways, and one way I can explicitly support this is to provide opportunities to learn from their experiences, including course exams.

Suggested References

Bowen, J. A. (2017). Teaching naked techniques: A practical guide to designing better classes. San Francisco, CA: Jossey-Bass.

Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick: The science of successful learning. Cambridge, Massachusetts: The Belknap Press of Harvard University.

Chen, P., Chavez, O., Ong, D. C., & Gunderson, B. (2017). Strategic resource use for learning: A self-administered intervention that guides self-reflection on effective resource use enhanced academic performance. Psychological Science, 28(6), 774-785. https://doi.org/10.1177/0956797617696456

Lovett, M. C. (2013). Make exams worth more than the grade: Using exam wrappers to promote metacognition. In M. Kaplan, N. Silver, D. LaVauge-

Manty & D. Meizlish (Eds.), Using reflection and metacognition to improve student learning: Across the disciplines, across the academy (pp. 18-52).  

Soicher, R. N., & Gurung, R. A. R. (2017). Do exam wrappers increase metacognition and performance? A single course intervention. Psychology Learning & Teaching, 16(1), 64-73. https://doi.org/10.1177/1475725716661872


Personal Characteristics Necessary for Metacognition

By Lauren Scharff, Ph.D., U. S. Air Force Academy *

            At my institution we have created the Science of Learning Team, a group of students who learn about the science of learning (including metacognition) and then lead seminars for other students who are hoping to improve their academic success. Additionally, as part of an ongoing scholarship of teaching and learning (SoTL) project, a small group of us (faculty and students) has assessed the efficacy of our various efforts to disseminate the science of learning to both faculty and students.

Students attending a seminar
Students attending a science of learning seminar

This past academic year I had the pleasure of working with Troy Mather, a senior who joined both the Science of Learning Team and the SoTL project effort as his capstone project.  Below are some of his final reflections regarding his experiences helping develop his peers’ metacognition and learning skills. I believe they provide some great insights regarding the personal characteristics necessary for metacognition. He also shares some personal applications that many of us might use as a model as we work to develop metacognition in our students.

What is metacognition and why is it important?

My personal definition for metacognition is having the awareness to self- regulate your learning approaches through modifications or corrections. Having the awareness to identify what you need to work on or change gives you the opportunity to grow. It does require modesty and humbleness to look at yourself and be motivated to change something you see as an area for growth. If you are someone like me, who isn’t someone naturally gifted with academics, metacognition is a tool that you can use to guide your growth as a student and learner.

What is the biggest challenge to developing student metacognition skills?

The biggest challenge I see with developing student metacognition skills is the fact that this skill is largely correlated with maturity. Time is a limiting factor because developing self-regulation doesn’t happen overnight. This makes teaching metacognition hard because you can tell others the definition of the concept and why it is important, but you can’t make them internalize the importance or change their behaviors. However, I have seen that most students eventually figure it out with time and maturity.

How can we overcome this challenge?

Something I found to help students get on that track of appreciating metacognition is by providing some personal examples of ways I have self-regulated my learning approaches and made clear improvements. Students listen to those moments of success and often feel more willing to make changes or even become more aware of what they should work on. Sometimes this goes outside of the academic environment. For example, one of the ways I have been most impacted by metacognition is with my training to be selected for Special Tactics/ Combat Rescue following my graduation.1

I told my students in our Science of Learning seminars that my training experience was a journey of self-reflection and deep accountability. Every day I had to have the self-awareness and honesty to identify my weak areas and do something about it. Some days I didn’t want to drown in the pool. Some days I didn’t want to run a marathon. And some days doing thousands of body weight exercises when I was already sore was a miserable thought. But, I pushed myself to do those things everyday because I knew if I didn’t, I wouldn’t reach my goal. I got a professional free- dive instructor and a track coach to help me with my training regimen. It was metacognition that allowed me to see areas to improve and reach out for resources.

With academics, students need to take advantage of all the resources they have in front of them. But, this requires self-accountability to make those identifications and be willing to put in that extra work. I told our students about my experience training for Special Operations because they hopefully saw someone with high ambitions and the willingness to put in the work. Every once in a while, learners need a motivational story to put them on track to accomplish their own goals. I have learned that metacognition is the start to achieving any level of greatness.

Using Troy’s Examples

Troy mentions humbleness and self-honesty as underlying characteristics of successful engagement in metacognition. That is not an aspect of metacognition that I have seen widely discussed, but it’s a great insight. It can be uncomfortable acknowledging aspects of our own efforts that have not been successful, and then examining them closely enough to come up with alternate strategies. This discomfort is especially strong if the alternate strategies appear to require more effort, and we’re not certain that they will lead us to success.

Many of our students face these uncomfortable moments on their path to become better learners. Perhaps we can help them through these uncomfortable barriers by more openly acknowledging the discomfort in facing one’s shortcomings, and letting students know that they are not alone in experiencing discomfort. Motivational stories such as the one Troy shared can help ease the resistance to being metacognitive. I’m sure we can all come up with a personal story or two that illustrate our own experiences as developing learners in some realm. Hopefully we can move past our discomfort in sharing our struggles in order to motivate our students to face their own struggles and self-regulate to move beyond them.

————————–

1 Special Tactics/ Combat Rescue is an elite team within the Special Operations career field  of the Air Force.

* Disclaimer: The views expressed in this document are those of the author and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.