Month: April 2019

Metacognitive Instruction: Suggestions for Faculty

by Audra Schaefer, Ph.D., Assistant Professor of Neurobiology & Anatomical Sciences, University of Mississippi Medical Center

Intro: This guest editor miniseries, “The Evolution of Metacognition”, examines metacognition of students at various stages of education (undergraduate, graduate and professional), so is fitting to wrap up this miniseries on the evolution of metacognition with discussion of faculty and metacognitive instruction. As educators we often find ourselves focused on enhancing the metacognition of our students.  Yet, in order for us to continue to improve/develop as teachers it is important for us to apply similar metacognitive approaches to our teaching.  This final post of the series will include my experiences in being a metacognitively aware educator and evidence-based suggestions for educators at any level who are looking to employ metacognition in their teaching. ~ Audra Schaefer, PhD, guest editor

————————————————————————————————

Being a teacher is not simple.  Teaching goes well beyond a content expert delivering information into the minds of students.  Harden and Crosby (2000) describe twelve roles of teachers that can be grouped into the broader roles of information provider, role model, facilitator, examiner, planner and resource developer. Effectively carrying out all of these roles can become a time consuming process, and while our focus is on our students, it’s important for us to step back and reflect on ourselves.  How effectively are we carrying out the various roles we manage in the process of teaching?

This seemingly straightforward question can be complicated to answer. Assessing yourself in the various roles encompassed by teaching may fall to the side when an instructor is unsure how to perform those roles in the first place. Educators in higher education often receive little to no explicit training in effective teaching methods and likely aren’t receiving much experience with learning theories.  So, naturally, instructors end up doing what they experienced as a student or mimicking what their colleagues are doing in class.  While this isn’t inherently problematic, simply doing it because that is what you experienced, or even because it works for someone else, is insufficient reasoning to keep doing it.

A metacognitive instructor asks why they are proceeding in a particular manner. How does this approach help you reach the goals you have for your course? How does it help your students achieve the objectives you’ve set for them? By the time an individual begins teaching at any level they’ve encountered plenty of situations that required them to apply problem-solving skills, be it in daily life or applying the scientific method to research. Why then as educators would we not apply the same concepts to our teaching?

What additional kinds of questions can, and perhaps should, metacognitive instructors be asking themselves?  If we consider that planning, monitoring and evaluating are all core components of metacognitive regulation (Schraw, 2006), these provide logical stages for instructors to reflect.  In my own experience, being reflective in each step is incredibly useful for improving the same session in future semesters, as well as preparing for other similar sessions.  The following are examples of questions I frequently ask myself, and if you’re interested in finding additional questions, Kimberly Tanner (2012) has a well-written article that provides numerous questions for faculty to ask students and to ask themselves in an effort to promote metacognition. 

When planning a course or class session, consider not only what your goals are for that course or session, but how you plan to reach those goals.  What do students already know about the topic for that session and how do you know what they know? 

In the middle of a class session it is useful to keep tabs on the session pace and to be adaptable in the moment to make improvements as you go.  What do I notice about student behavior in a given session and what might be the cause?  After a session, and after a course is completed I also take the time to think about how I want to change things and why I want to change them in the future. A common thread here being the “why”.  Why do I do what I do? Why did the students respond the way they did? Why would I make a particular change?

the word "Why" written using question marks

Finally, I’ve found modeling metacognitive behaviors to be quite useful with my students (Tanner, 2012).  When discussing challenging topics with students I make a deliberate effort to think about what aspects of that topic gave me difficulty when I first learned it, and then I make a point to explain that topic to the students in a way that helped me (or past students) make sense of it. 

For example, in neuroanatomy there are numerous pathways in the brain and spinal cord that become confusing very quickly.  I frequently teach these by creating the same simplified drawings I made as a student to sort out these pathways.  I also encourage students to draw with me in class to engage them during a lecture, and numerous students have commented that they begin drawing on their own while studying, an approach that they had not previously used.

I also make a point to model how to proceed when faced with a limitation in my knowledge. This has become an important goal for me in my teaching because I’ve noticed many students struggle to recognize what they do and do not know, or struggle with how to proceed when they don’t know something. Earlier in this miniseries, Dr. Husmann and Dr. Hoffman each provided examples of undergraduate and medical students struggling with metacognition, demonstrating that these difficulties span multiple educational levels. We can shape our instruction to students of any educational stage to gain a sense of their current skill level(s) and adjust our teaching to help improve those skills.

In class, when I am inevitably asked a question to which I do not have an answer, I own it and don’t make a big deal out of it. I share what I do know and then proceed to either have them help me research the answer, or I follow up with them to be sure that we’ve all learned from it.  Although I haven’t explicitly assessed whether it’s having any effect on students, I am getting little bits of data to suggest there are at least a few students who’ve noticed.  One of my favorites is this comment from a medical student after completing a neuroscience course I taught, “If she has taught me anything (besides a lot of Neuro) is that it is okay to not know the answer, we aren’t always going to know everything and it’s completely okay.” 

As educators, at any level, we serve as role models. If we expect our students to become better at regulating their learning, we should expect the same of ourselves. In the first post of this miniseries Caroline Mueller discussed how as a graduate student she is working to implement metacognitive approaches to both her learning and teaching. Regardless of your experience with teaching, metacognitive instruction can help you continue to improve. Our attitudes and mindset are important for setting the tone for a classroom environment. If we aim to help our students develop their metacognitive skills, we should aim to do so ourselves.

Harden, R.M., & J. Crosby. (2000). AMEE Guide No 20: The good teacher is more than a lecturer – the twelve roles of the teacher. Medical Teacher, 22:4, 334-347.

Schraw, G., Crippen, K.J., & K. Hartley. (2006). Promoting self-regulation in science education: Metacognition as a part of a broader perspective on learning. Research in Science Education, 36, 111-139.

Tanner, K.D. (2012). Promoting Student Metacognition. CBE – Life Sciences Education, 11, 113-120. [https://www.improvewithmetacognition.com/promoting-student-metacognition/]


Enhancing Medical Students’ Metacognition

by Leslie A. Hoffman, PhD, Assistant Professor of Anatomy & Cell Biology, Indiana University School of Medicine – Fort Wayne

The third post in this guest editor miniseries examines how metacognition evolves (or doesn’t) as students progress into professional school.  Despite the academic success necessary to enter into professional programs such as medical school or dental school, there are still students who lack the metacognitive awareness/skills to confront the increased academic challenges imposed by these professional programs.  In this post Dr. Leslie Hoffman reflects on her interactions with medical students and incorporates data she has collected on self-directed learning and student reflections on their study strategies and exam performance. ~Audra Schaefer, PhD, guest editor

————————————————————————————————-

The beginning of medical school can be a challenging time for medical students.  As expected, most medical students are exceptionally bright individuals, which means that many did not have to study very hard to perform well in their undergraduate courses.  As a result, some medical students arrive in medical school without well-established study strategies and habits, leaving them overwhelmed as they adjust to the pace and rigor of medical school coursework.  Even more concerning is that many medical students don’t realize that they don’t know how to study, or that their study strategies are ineffective, until after they’ve performed poorly on an exam.  In my own experience teaching gross anatomy to medical students, I’ve found that many low-performing students tend to overestimate their performance on their first anatomy exam (Hoffman, 2016).  In this post I’ll explore some of the reasons why many low-performing students overestimate their performance and how improving students’ metacognitive skills can help improve their self-assessment skills along with their performance.

Metacognition is the practice of “thinking about thinking” that allows individuals to monitor and make accurate judgments about their knowledge, skills, or performance.  A lack of metacognitive awareness can lead to overconfidence in one’s knowledge or abilities and an inability to identify areas of weakness.  In medicine, metacognitive skills are critical for practicing physicians to monitor their own performance and identify areas of weakness or incompetence, which can lead to medical errors that may cause harm to patients.  Unfortunately, studies have found that many physicians seem to have limited capacity for assessing their own performance (Davis et al., 2006).  This lack of metacognitive awareness among physicians highlights the need for medical schools to teach and assess metacognitive skills so that medical students learn how to monitor and assess their own performance. 

Cartoon of a brain thinking about a brain

In my gross anatomy course, I use a guided reflection exercise that is designed to introduce metacognitive processes by asking students to think about their study strategies in preparation for the first exam and how they are determining whether those strategies are effective.   The reflective exercise includes two parts: a pre-exam reflection and a post-exam reflection.  

The pre-exam reflection asks students to identify the content areas in which they feel most prepared (i.e. their strengths) and the areas in which they feel least prepared (i.e. their weaknesses).  Students also discuss how they determined what they needed to know for the upcoming exam, and how they went about addressing their learning needs.  Students were also asked to assess their confidence level and make a prediction about their expected performance on the upcoming exam.  After receiving their exam scores students completed a post-exam reflection, which asked them to discuss what, if any, changes they intended to make to their study strategies based on their exam performance. 

My analysis of the students’ pre-exam reflection comments found that the lowest performing students (i.e. those who failed the exam) often felt fairly confident about their knowledge and predicted they would perform well, only to realize during the exam that they were grossly underprepared.  This illusion of preparedness may have been a result of using ineffective study strategies that give students a false sense of learning.  Such strategies often included passive activities such as re-watching lecture recordings, re-reading notes, or looking at flash cards.  In contrast, none of the highest performing students in the class over-estimated their exam grade; in fact, many of them vastly underestimated their performance. A qualitative analysis of students’ post-exam reflection responses indicated that many of the lowest performing students intended to make drastic changes to their study strategies prior to the next exam.  Such changes included utilizing different resources, focusing on different content, or incorporating more active learning strategies such as drawing, labeling, or quizzing.  This suggests that the lowest performing students hadn’t realized that their study strategies were ineffective until after they’d performed poorly on the exam.  This lack of insight demonstrates a deficiency in metacognitive awareness that is pervasive amongst the lowest performing students and may persist in these individuals beyond medical school and into their clinical practice (Davis et al., 2006).

So how can we, as educators, improve medical students’ (or any students’) metacognitive awareness to enable them to better recognize their shortcomings before they perform poorly on an exam?  To answer this question, I turned to the highest performing students in my class to see what they did differently.  My analysis of reflection responses from high-performing students found that they tended to monitor their progress by frequently assessing their knowledge as they were studying.  They did so by engaging in self-assessment activities such as quizzing, either using question banks or simply trying to recall information they’d just studied without looking at their notes.  They also tended to study more frequently with their peers, which enabled them to take turns quizzing each other.  Working with peers also provided students with feedback about what they perceived to be the most relevant information, so they didn’t get caught up in extraneous details. 

The reflective activity itself is a technique to help students develop and enhance their metacognitive skills.  Reflecting on a poor exam performance, for example, can draw a student’s attention to areas of weakness that he or she was not able to recognize, or ways in which his or her preparation may have been inadequate.   Other techniques for improving metacognitive skills include the use of think-aloud strategies in which learners verbalize their thought process to better identify areas of weakness or misunderstanding, and the use of graphic organizers in which learners create a visual representation of the information to enhance their understanding of relationships and processes (Colbert et al., 2015). 

Ultimately, the goal of improving medical students’ metacognitive skills is to ensure that these students will go on to become competent physicians who are able to identify their areas of weakness, create a plan to address their deficiencies, and monitor and evaluate their progress to meet their learning goals.   Such skills are necessary for physicians to maintain competence in an ever-changing healthcare environment.

Colbert, C.Y., Graham, L., West, C., White, B.A., Arroliga, A.C., Myers, J.D., Ogden, P.E., Archer, J., Mohammad, S.T.A., & Clark, J. (2015).  Teaching metacognitive skills: Helping your physician trainees in the quest to ‘know what they don’t know.’  The American Journal of Medicine, 128(3), 318-324.

Davis, D.A., Mazmanian, P.E., Fordis, M., Harrison, R., Thorpe, K.E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296, 1094-1102. Hoffman, L.A. (2016). Prediction, performance, and adjustments: Medical students’ reflections on the first gross anatomy exam.  The FASEB Journal 30 (1 Supplement): 365.2.


Metacognition v. pure effort: Which truly makes the difference in an undergraduate anatomy class?

by Polly R. Husmann, Ph.D., Assistant Professor of Anatomy & Cell Biology, Indiana University School of Medicine – Bloomington

Intro: The second post of “The Evolution of Metacognition” miniseries is written by Dr. Polly Husmann, and she reflects on her experiences teaching undergraduate anatomy students early in their college years, a time when students have varying metacognitive abilities and awareness.  Dr. Husmann also shares data collected that demonstrate a relationship between students’ metacognitive skills, effort levels, and final course grades. ~ Audra Schaefer, PhD, guest editor

————————————————————————————————–

I would imagine that nearly every instructor is familiar with the following situation: After the first exam in a course, a student walks into your office looking distraught and states, “I don’t know what happened.  I studied for HOURS.”  We know that metacognition is important for academic success [1, 2], but undergraduates often struggle with how to identify study strategies that work or to determine if they actually “know” something.  In addition to metacognition, recent research has also shown that repeated recall of information [3] and immediate feedback also improve learning efficiency [4].  Yet in large, content-heavy undergraduate classes both of these goals are difficult to accomplish.  Are there ways that we might encourage students to develop these skills without taking up more class time? 

Online Modules in an Undergraduate Anatomy Course

I decided to take a look at this through our online modules.  Our undergraduate human anatomy course (A215) is a large (400+) course mostly taken by students planning to go into the healthcare fields (nursing, physical therapy, optometry, etc.).  The course is comprised of both a lecture (3x/week) and a lab component (2x/week) with about forty students in each lab section.  We use the McKinley & O’Loughlin text, which comes with access to McGraw-Hill’s Connect website.  This website includes an e-book, access to online quizzes, A&P Revealed (a virtual dissection platform with images of cadavers) and instant grading.  Also available through the MGH Connect site are LearnSmart study modules. 

These modules were incorporated into the course along with the related electronic textbook as optional extra credit assignments about five years ago as a way to keep students engaging with the material and (hopefully) less likely to just cram right before the tests. Each online module asks questions over a chapter or section of a chapter using a variety of multiple-choice, matching, rank order, fill-in-the-blank, and multiple answer questions. For each question, students are not only asked for their answer, but also asked to rank their confidence for their answer on a four-point Likert scale. After the student has indicated his/her confidence level, the module will then provide immediate feedback on the accuracy of their response. 

During each block of material (4 total blocks/semester) in our anatomy course during the Fall 2017 semester, 4 to 9 LearnSmart modules were available and 2 were chosen by the instructor after the block was completed to be included for up to two points of extra credit (total of 16 points out of 800).  Given the frequency of the opening scenario, I decided to take a look at these data and see what correlations existed between the LearnSmart data and student outcomes in our course.

Results

The graphs (shown below) illustrated that the students who got As and Bs on the first exam had done almost exactly the same number of LearnSmart practice questions, which was nearly fifty more questions than the students who got Cs, Ds, or Fs.  However, by the end of the course the students who ultimately got Cs were doing almost the exact same number of practice questions as those who got Bs!  So they’re putting the same effort into the practice questions, but where is the problem? 

The big difference is seen in the percentage of these questions for which each group was metacognitively aware (i.e., accurately confident when putting the correct answer or not confident when putting the incorrect answer).  While the students who received Cs were answering plenty of practice questions, their metacognitive awareness (accuracy) was often the worst in the class!  So these are your hard-working students who put in plenty of time studying, but don’t really know when they accurately understand the material or how to study efficiently. 

Graphs showing questions completed as well as accuracy of self-assessment.

The statistics further confirmed that both the students’ effort on these modules and their ability to accurately rate whether or not they knew the answer to a LearnSmart practice question were significantly related to their final outcome in the course. (See right-hand column graphs.) In addition to these two direct effects, there was also an indirect effect of effort on final course grades through metacognition.  So students who put in the effort through these practice questions with immediate feedback do generally improve their metacognitive awareness as well.  In fact, over 30% of the variation in final course grades could be predicted by looking at these two variables from the online modules alone.

Flow diagram showing direct and indirect effects on course grade

Effort has a direct effect on course grade while also having an indirect effect via metacognition.

Take home points

  • Both metacognitive skills (ability to accurately rate correctness of one’s responses) and effort (# of practice questions completed) have a direct effect on grade.
  • The direct effect between effort and final grade is also partially mediated by metacognitive skills.
  • The amount of effort between students who get A’s and B’s on the first exam is indistinguishable.  The difference is in their metacognitive skills.
  • By the end of the course, C students are likely to be putting in just as much effort as the A & B students; they just have lower metacognitive awareness.
  • Students who ultimately end up with Ds & Fs struggle to get the work done that they need to.  However, their metacognitive skills may be better than many C level students.

Given these points, the need to include instruction in metacognitive skills in these large classes is incredibly important as it does make a difference in students’ final grades.  Furthermore, having a few metacognitive activities that you can give to students who stop into your office hours (or e-mail) about the HOURS that they’re spending studying may prove more helpful to their final outcome than we realize.

Acknowledgements

Funding for this project was provided by a Scholarship of Teaching & Learning (SOTL) grant from the Indiana University Bloomington Center for Innovative Teaching and Learning.  Theo Smith was instrumental in collecting these data and creating figures.  A special thanks to all of the students for participating in this project!

References

1. Ross, M.E., et al., College Students’ Study Strategies as a Function of Testing: An Investigation into Metacognitive Self-Regulation. Innovative Higher Education, 2006. 30(5): p. 361-375.

2. Costabile, A., et al., Metacognitive Components of Student’s Difficulties in the First Year of University. International Journal of Higher Education, 2013. 2(4): p. 165-171.

3. Roediger III, H.L. and J.D. Karpicke, Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Psychological Science, 2006. 17(3): p. 249 – 255.

4. El Saadawi, G.M., et al., Factors Affecting Felling-of-Knowing in a Medical Intelligent Tutoring System: the Role of Immediate Feedback as a Metacognitive Scaffold. Advances in Health Science Education, 2010. 15: p. 9-30.


Learning about learning: A student perspective

by Caroline Mueller, B.S., Clinical Anatomy PhD student, University of Mississippi Medical Center

Intro: In this guest editor miniseries, “The Evolution of Metacognition”, we will be discussing a progression of metacognitive awareness and development of metacognition in multiple stages of education, from undergraduate, to graduate and professional students, and even faculty. In this first post Caroline Mueller, a doctoral student in an anatomy education program, is providing a student perspective.  She shares reflections on learning about metacognition, how it has shaped her approaches to learning, and how it is influencing her as an emerging educator.  ~Audra Schaefer, PhD, guest editor

———————————————————————————————-

As a second-year graduate student hearing the word “metacognition” for the first time, I thought the idea of “thinking about thinking” seemed like another activity necessitated by teachers to take up more time. After looking into what metacognition actually meant and the processes it entails, my mindset changed. It is logical to think about the thought processes that occur during learning. Engaging in metacognitive thought seems like an obvious, efficient activity for students to do to test their knowledge—yet very few do it, myself included. In undergrad, I prided myself on getting high grades, thinking that my method of reading, re-writing, memorizing, and then repeating was a labor-intensive but effective method. It did the job, and it resulted in high grades. However, if my goals included retaining the content, this method failed me. If someone today asked me about the Krebs Cycle, I could not recite it like I could for the test, and I definitely could not tell you about its function (something to do with glucose and energy?).

Upon entering graduate school, what I thought were my “fool-proof” methods of study soon became insufficient and fallible. The work load in medical gross anatomy and medical histology increased by at least 20 times (well, it felt like it anyway). It was laborious to keep up with taking notes in lecture, re-writing, reading the text, and then testing myself with practice questions. I felt as though I was drowning in information, and I saw a crippling arthritis in my near future. I then faced my first devastating grade. I felt cheated that my methods did not work, and I wondered why. Needing a change, I started trying different study methods. I started reviewing the information, still re-writing, but self-quizzing with a small group of classmates instead of by myself. We would discuss what we got wrong and explain answers if we knew them. It helped me improve my grades, but I wish I had more guidance about metacognition at that point.

As I begin studying for my terrifying qualifying exams this semester, I am currently facing the daunting task of studying all the material I have learned in the last 2 years of graduate school. Easy task, right? Even though you may sense my dread, I have a different approach to studying because of what I’ve recently learned about metacognition. An important aspect of metacognition is self-assessment, using tools such as pre-assessment and the most confusing point (muddiest point). The pre-assessment is a tool that allows students to examine their current understanding of a topic and to direct them to think about what they do and do not know. It helps guide students to focus their efforts on those elements they do not know or understand well (Tanner, 2012). The muddiest point tool can be used at the end of a long day of studying. Students reflect on the information covered in a class or study session and assess what was the muddiest point (Tanner, 2012).

Both tools have shaped my approach to studying.  Now I study by human body systems, starting each system off by writing what I do know about the subject and then writing down what I want to know by the end of my review. This aids in my assessment of what I do and do not know, so that I can orient myself to where I struggle the most. At first, it seemed like a time-intensive activity, but it quickly made me realize that it was more efficient then rewriting and rereading the content I already knew. I implemented muddiest point in my studies too because after a strenuous day of trying to grasp intense information, I end up feeling like I still do not know anything. After reviewing the information and filling in the gaps, at the end of my week of review, I quiz myself and ask myself what I was most confusing. It helps me plan for future study sessions.

Metacognition feels like it takes a lot of time when you first start doing it because it makes the learner deal with the difficult parts of a subject matter. Students, myself included, want the act of acquiring new information to be rewarding, quick, and an affirmation of their competency of the material. An example of this is when I would get an answer correct when I did practice questions while preparing for an exam, but I never thought about why the correct answer was correct. Getting it right could have been pure luck; in my mind, I must have known the material. By thinking about the “why,” it prompts students to think deeply about their thought process to picking that answer. This act alone helps solidify understanding of the topic. If one can explain how they got to the answer, or why they believe an answer to be true, it allows them to assess how well they understand the content matter.

cartoon of a brain working out using books as weights

My role as a student is beginning to change—I have become a teacher’s assistant, slowly on my way to full-on teacher status. After learning about metacognition and applying it as a student, I attempted to try it on the students I teach.

For example, an important part of metacognition is learning to recognize what you do and do not know. In anatomy lab, in order to prompt students to think deeper about material, I ask students what they know, rather than just giving them the answer to their questions. I let them describe the structure and ask them to explain why they think that structure is what it is.

When I first did this, students resisted—the stress of the first-year medical school makes students desire the answer immediately and to move on. But I persisted in asking questions, explaining to students that finding out what you do know and do not know allows you to focus your studying to filling in those gaps.

Since I am a new convert to teacher assistant from student, students often ask me the best ways to study and about how I studied. I again urge them to take an approach that helps identify gaps in their knowledge. I encourage them to go over the chapter headings and write down what they know about each one, essentially completing a preassessment I previously mentioned.

At this point, I might be a little rough in my approach to instill the incredible power of metacognitive skills in students, but I am still working out the kinks. I am still learning—learning to be an effective teacher, learning the content as a student, and learning to learn about teaching and learning. As a student and a teacher, my hope for the future of my teaching is that I learn how to implement metacognitive methods effectively and to be able to assess these methods and keep trying to improve on them.

Tanner, K.D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11, 113-120. [https://www.improvewithmetacognition.com/promoting-student-metacognition/]