Comprehension Monitoring: The Role of Conditional Knowledge Part 2

by Antonio Gutierrez, Georgia Southern University

In my previous post, I discussed the role conditional knowledge (i.e., the why, when, and where to apply strategies given task demands) plays in learners’ ability to calibrate their performance against their actual performance. This is in light of debates about the relations between the various dimensions of metacognition. Calibration is a component of one’s ability to monitor comprehension, which is a regulatory function. Conditional knowledge, on the other hand, is part of the knowledge component of metacognition. As a researcher I often wonder whether instead of making assumptions that these various metacognitive functions are related whether perhaps we should empirically test these assumptions. In metacognitive research it is often assumed that the knowledge and regulation aspects of metacognition are related. From a theoretical perspective, this makes sense. However, for us to assume that this is the case with all samples and populations may be a stretch, especially given the diversity and individual differences among learners. In this vein, I am currently seeking ethics approval to conduct research with middle school students because this is an understudied population with respect to metacognition. In this proposed research I plan to not only investigate calibration among middle school students and the influence metacognitive strategy training has on learners’ calibration, but I plan to empirically assess the association between the eight dimensions of metacognition (Knowledge: declarative, procedural, and conditional; Regulation: planning, information management, debugging strategies; comprehension monitoring; and evaluation of learning). I will also attempt to test the predictive power of various components of metacognition on learners’ calibration. I am especially interested in empirically measuring the association between conditional knowledge and calibration as well as the predictive power of conditional knowledge on calibration. I expect that metacognitive strategy training will improve learners’ performance, confidence judgments, and also their calibration. I also suspect that those with greater conditional knowledge will have better calibration, and hence, I expect conditional knowledge to strongly predict calibration.

This particular study is one among a series of scientific investigations on the validity of theoretical claims made when researchers discuss metacognition. In my attempt to provide educators a toolkit of domain-general metacognitive strategies they can readily apply in their classrooms, this series of studies will help me provide the empirical evidence necessary to demonstrate the utility and relevance of metacognitive strategies to not only scholars but practitioners as well. These strategies have been adapted from an unpublished pilot study I conducted prior to my dissertation. This research will help me to continue to refine these strategies to better suit adolescents. Moreover, it will shed some light on the link between conditional metacognitive knowledge and calibration, which is a topic that began with earlier posts. Stay tuned for preliminary results of my first investigation.


A Mindfulness Perspective on Metacognition

by Chris Was, Kent State University

If you have any interest in metacognition, you have likely come across the description of metacognition as thinking about one’s thinking. A number of posts to this blog (including my own) provide evidence to support the conclusion that metacognition can be “learned” and improved. Further, improved metacognition leads to improve self-regulation and positive academic outcomes. There is also a good deal of evidence that training in mindfulness improves cognitive function and attention (e.g., Chambers, Lo, & Allen, 2008). Flook, et al (2010) found that mindfulness-training program improved executive functions in young elementary school students. Zeidan, et al (2010) found that mindfulness training improved executive function and metacognitive insight. This post will focus on the relationship between metacognition and mindfulness.

Let me preface by stating that mindfulness need not refer to esoteric religious beliefs, but it is often defined as a mental state achieved by focusing one’s awareness on the present moment and acknowledging one’s feelings, thoughts and bodily sensations. Kabat-Zinn (1990) describes mindfulness as bringing attention to moment-to-moment experience. In my own work on metacognitive knowledge monitoring, I have required students to make moment-to-moment (more accurately, item-by-item) judgments of their knowledge.. Hypotheses, such as cue-familiarity, provide reasonable explanations for how students and research participants rate feelings of knowing (FOK), judgments of learning (JOLs), judgments of knowing (JOK), etc. However, the simple fact is one must attend to these feelings and thoughts to provide a judgment. In psychological and educational literature, we refer to one using metacognition to make these judgments. Clinical psychology programs such as mindfulness based stress reduction (MBSR) and cognitive behavior therapy (CBT) refer to the patient/participant as being mindful about their emotions, thoughts, and actions. Although the majority of research and application of mindfulness has occurred in clinical settings, there is a great deal of potential in examining the relationship between mindfulness and metacognition.

It isn’t clear how metacognition and mindfulness are related. Some argue that metacognition is not mindful because a true expert in mindfulness does not need to reflect upon his or her thinking, but only to attend to what they are presently doing. I am not convinced. Much of the work on metacognitive improvement has focused on semester long training to improve students knowledge monitoring. The mindfulness research has focused on training students to focus on their moment-to-moment experiences and thoughts. Clearly, there is a relationship between metacognition and executive function, but I have yet to see evidence that training in one improves the other.

One argument made to dissociate mindfulness from metacognition is that metacognitive processes are by necessity reflective or retrospective and that truly being mindful does not require reflection. For example, for a student to practice metacognition during study, she must ask herself, “Do I understand this concept?” Then, depending on the answer the student may or may not adjust the cognitive actions in which she is engaged to learn. This cycle is simply explained by the Nelson and Narens (1990). Now let’s think about a practitioner of mindfulness meditation. While meditating he may chose to focus his attention on the breath. Noticing when he is breathing in and noticing when he is breathing out. During this practice his mind may wander (this is true of even the most practiced at meditation). When this happens, he will gently bring his attention back to the breath. This process, just like that of the studying student, requires one to observe the cognitive processes and exert control over those processes when necessary. This to fits nicely into the metacognitive model offered by Nelson and Narens.

Imagine you are reading a novel on summer vacation. The book is enjoyable, but not a challenging read. Your are enjoying the sun and the sounds on the beach as you read, but suddenly notice you have not really attended to the last couple of pages and are not sure what has transpired in the plot. You choose to reread the last couple of pages and pay more attention. Imagine now you are a student. You are reading a very dull textbook chapter with the TV on and your smart phone near by. A student with little metacognitive resources (whether it be due working memory capacity, attentional control, executive function, etc.) is likely to mind wander (Hollis & Was, 2014). Students in my classes have often told me the hardest part of studying is staying focused, even when the topic is of interest. Ben Hollis and I found that students watching a video as part of an online course were often distracted by thoughts of checking the social media outlets. Not distracted by checking, but just thought of checking them. What if students were practiced at focusing attention, noticing when their minds wander and bringing the attention back to the task at hand?

It seems to me that if metacognition is knowledge and control of one’s cognitive processes and training in mindfulness increases one’s ability to focus and control awareness in a moment-by-moment manner, then perhaps we should reconsider, and investigate the relationship between mindfulness and metacognition in education and learning.

 

References

Bishop, S. R., Lau, M., Shapiro, S., Carlson, L., Anderson, N. D., Carmody, J., Segal, Z. V.,    Abbey, S., Speca, M., Velting, D. and Devins, G. (2004), Mindfulness: A Proposed Operational Definition. Clinical Psychology: Science and Practice,     11: 230–241. doi: 10.1093/clipsy.bph077

Chambers, R., Lo, B. C. Y., & Allen, N. B. (2008). The impact of intensive mindfulness training on attentional control, cognitive style, and affect. Cognitive Therapy and Research32(3), 303-322.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American psychologist34(10), 906.

Flook, L., Smalley, S. L., Kitil, M. J., Galla, B. M., Kaiser-Greenland, S., Locke, J., … &  Kasari, C. (2010). Effects of mindful awareness practices on executive functions in elementary school children. Journal of Applied School         Psychology,26(1), 70-95.

Kabat-Zinn, J. (1990). Full catastrophe living: Using the wisdom of your mind to face    stress, pain and illness. New York : Dell.

Zeidan, F., Johnson, S. K., Diamond, B. J., David, Z., & Goolkasian, P. (2010).  Mindfulness meditation improves cognition: evidence of brief mental training.Consciousness and cognition19(2), 597-605.


Just-in-Time for Metacognition

By John Draeger, SUNY Buffalo State

This post brings metacognition to an already valuable teaching tool. Just-in-time techniques require that students submit short assignments prior to class. Instructors review those answers before class and use them to shape class time. In my philosophy classes, for example, I assign two short questions via a course management system (e.g., Blackboard). At least one of the questions is directly related to the reading. Students are required to submit their answers electronically by 11:00 p.m. the night before class. When I wake up in the morning, I read through their responses and use them to make decisions about how class time will be used. If students seemed to grasp the reading, then I spend less time reviewing the basic arguments and more time exploring deeper content and connections. If student responses displayed a misunderstanding of the reading, then we spend class time carefully examining passages in the text and digging out the relevant arguments.

Just-in-Time techniques have been used in a variety of disciplines and they have been shown to increase the likelihood that students will complete their reading assignments, read more carefully, and take ownership over their learning (Novak 1999; Simkins & Maier, 2009; Schraff et al. 2011). However, just-in-time assignments are typically used to prompt students to complete their assigned reading pages and gauge their basic comprehension. While both are valuable, I argue that the technique can also be used to promote other important skills.

For example, pre-class questions can be used to develop higher-order thinking skills. Students can be asked to examine an author’s point of view, underlying assumptions, or the implications of her view. Such questions prompt students to move beyond their knowledge of what is contained in the text towards active engagement with that text. Students can be asked to apply concepts in the reading (e.g., stereotype bias) to something in the news. And students can be asked to analyze the connections between related course ideas. In a previous post, “Using metacognition to uncover the substructure of moral issues,” I argued that students begin to “think like a philosopher” when they can move beyond the surface content (e.g., hate speech and national security) and towards the underlying philosophical substructure (e.g., rights, well-being, dangers of governmental intrusion). Like other skills, developing higher-order thinking skills requires practice. Because just-in-time assignments are a regular part of a student’s week, incorporating high-order thinking questions into just-in-time assignments can give students regular opportunities to practice and hone those skills.

Likewise, pre-class assignments can give students a regular outlet to practice and develop metacognition. Students can be asked to reflect on how they prepared for class and whether it was effective (Tanner 2012). Pre-class questions might include: how long did you spend with the reading? Did you finish? Did you annotate the text? Did you write a summary of the central argument? Did you formulate questions based on the reading for class discussion? Was this reading more difficult than the previous? If so, why? Did you find yourself having an emotional reaction to the reading? If so, did this help or hinder your ability to understand the central argument? Are your reading techniques adequately preparing you for class? Or, are you finding yourself lost in class discussion despite having spent time doing the reading? If pre-class questions related to higher-order thinking ask students to do more than simply “turn the pages,” then pre-class questions related to metacognition ask students to do more than simply engage with the material, but also engage with their own learning processes.

When just-in-time questions are a regular part of the ebb and flow of a course, students must regularly demonstrate how much they know and instructors can regularly use that information to guide course instruction. These techniques work because there is a consistent accountability measure built-in. I suggest that just-in-time assignments can also be used to give students regular practice developing both higher-order thinking and metacognition skills. I have been incorporating higher-ordering thinking into just-in-time assignments for years, but I confess that I have only given metacognition prompts when things have “gone wrong” (e.g., poor performance on exams, consistent misunderstanding of the reading). Responses to these questions have led to helpful conversation about the efficacy of various learning methods. Writing this blog post has prompted me to see the potential benefits of asking such questions more often. I pledge to do just that and to let you know how my students respond.

 

References

Novak, G., Patterson, E., Gavrin, A., & Christian, W. (1999). Just-in-time teaching: Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall.

Scharff, L., Rolf, J. Novotny, S. and Lee, R. (2011). “Factors impacting completion of pre-class assignments (JiTT) in Physics, Math, and Behavioral Sciences.” In C. Rust (ed.), Improving Student Learning: Improving Student Learning Global Theories and Local Practices: Institutional, Disciplinary and Cultural Variations. Oxford Brookes University, UK.

Simkins, S. & Maier, M. (2009). Just-in-time teaching: Across the disciplines, across the academy. Stylus Publishing, LLC..

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education11(2), 113-120.


Despite Good Intentions, More is Not Always Better

by Lauren Scharff, U.S. Air Force Academy*

A recent post to the PSYCHTEACH listserv got me thinking about my own evolution as a teacher trying my best to help the almost inevitable small cluster of students who struggled in my courses, often despite claiming to “have studied for hours.” The post asked “Have any of you developed a handout on study tips/skills that you give to your students after the first exam?” A wide variety of responses were submitted, all of which reflected genuinely good intentions by the teachers.

However, based on my ongoing exploration of metacognition and human learning, I believe that, despite the good intentions, some of the recommendations will not consistently lead to the desired results. Importantly, these recommendations actually seem quite intuitive and reasonable on the surface, which leads to their appeal and continued use. Most of those that fall into this less ideal category do so because they imply that “More is Better.”

For example, one respondent shared, “I did correlations of their test scores with their attendance so far, the number of online quizzes they have taken so far, and the combined number of these two things. [All correlations were positive ranging from 0.35 to 0.57.] So I get to show them how their behaviors really are related to their scores…”

This approach suggests several things that all seem intuitively positive: online quizzes are a good way to study and attending class will help them learn. I love the empowerment of students by pointing out how their choice of behaviors can impact their learning! However, the message that more quizzes and simple attendance will lead to better grades does not capture the true complexity of learning.

Another respondent shared a pre-post quiz reflection assignment in which some of the questions asked about how much of the required reading was completed and how many hours were put into studying. Other questions asked about the use of chapter outcomes when reading and studying, the student’s expected grade on the quiz, and an open-ended question requesting a summary of study approaches.

This pre-post quiz approach seems positive for many reasons. Students are forced to think about and acknowledge levels and types of effort that they put into studying for the quizzes. There is a clear suggestion that using the learning outcomes to direct their studying would be a positive strategy. They are asked to predict their grades, which might help them link their studying efforts with predicted grades. These types of activities are actually good first steps at helping students become more metacognitive (aware and thoughtful) about their studying. Yea!

However, a theme running through the questions seems to be, again, “more is better.” More hours. More reading. The hidden danger is that students may not know how to effectively use the learning outcomes, how to read, how to effectively engage during class, how to best take advantage of practice quizzes to promote self-monitoring of learning, or what to do during those many hours of studying.

Thus, the recommended study strategies may work well for some students, but not all, due to differences in how students implement the strategies. Therefore, even a moderately high correlation between taking practice quizzes and exam performance might mask the fact that there are subgroups for which the results are less positive.

For example, Kontur and Terry (2013) found the following in a core Physics course, “On average, completing many homework problems correlated to better exam scores only for students with high physics aptitude. Low aptitude physics students had a negative correlation between exam performance and completing homework; the more homework problems they did, the worse their performance was on exams.”

I’m sure you’re all familiar with students who seem to go through “all the right motions” but who still struggle, become frustrated, and sometimes give up or develop self-doubt about their abilities. Telling students to do more of what they’re already doing if it’s not effective will actually be more harmful.

This is where many teachers feel uncomfortable because they are clearly working outside their disciplines. Teaching students how to read or how to effectively take notes in class, or how to self-monitor their own learning and adjust study strategies to different types of learning expectations is not their area of expertise. Most teachers somehow figured out how to do these things well on their own, or they wouldn’t be teachers now. However, they may never have thought about the underlying processes of what they do when they read or study that allowed them to be successful. They also feel pressures to cover the disciplinary content and focus on the actual course material rather than learning skills. Unfortunately, covering material does little good if the students forget most of the content anyway. Teaching them skills (e.g., metacognitive study habits) offers the prospect of retaining more of the disciplinary content that is covered.

The good news is that there are more and more resources available for both teachers and students (check out the resources on this website). A couple great resources specifically mentioned by the listserv respondents are the How to Get the Most out of Studying videos by Stephen Chew at Samford University and the short reading (great to share with both faculty and students) called The Six Hour D… and How to Avoid it by Dewey (1997). Both of these highlighted resources focus on metacognitive learning strategies.

This reflection on the different recommendations is not meant to belittle the well-intentioned teachers. However, by openly discussing these common suggestions, and linking to what we know of metacognition, I believe we can increase their positive impact. Share your thoughts, favorite study suggestions and metacognitive activities by using the comments link below, or submitting them under the Teaching Strategies tab on this website.

References

Dewey, R. (1997, February 12) The “6 hour D” and how to avoid it. [Online]. Available: http://www.psywww.com/discuss/chap00/6hourd.htm.

Kontur, F. & Terry, N. The benefits of completing homework for students with different aptitudes in an introductory physics course. Cornell Physics Library Physics Education. arXiv:1305.2213

 

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


Negotiating Chaos: Metacognition in the First-Year Writing Classroom

by Amy Ratto Parks, Composition Coordinator/Interim Director of Composition, University of Montana

“Life moves pretty fast. If you don’t stop and look around once in a while, you could miss it.” John Hughes, Ferris Bueller’s Day Off

Although the movie Ferris Bueller’s Day Off (Hughes, 1986) debuted long before our current first-year college students were born, the combined sentiment of the film remains relevant to them. If we combined Ferris’ sense of exuberant freedom with Cameron’s grave awareness of personal responsibility, and added Sloane’s blasé ennui we might see an accurate portrait of a typical first-year student’s internal landscape. Many of our students are thrilled to have broken out of the confines of high school but are worried about not being able to succeed in college, so they arrive in our classrooms slumped over their phones or behind computer screens, trying to seem coolly disengaged.

The life of the traditional first-year student is rife with negotiations against chaos. Even if we remove the non-academic adjustments of living away from home, their lives are full of confusion. All students, even the most successful, will likely find their learning identities challenged: what if all of their previous academic problem-solving strategies are inadequate for the new set of college-level tasks?

In the first-year writing classroom, we see vivid examples of this adjustment period play out every year. Metacognitive activities like critical reflective writing help students orient themselves because they require students to pause, assess the task at hand, and assess their strategies for meeting the demands of the task. Writing studies researchers know that reflection benefits writers (Yancey, 1998) and portfolio assessment, common in first-year program across the country, emphasizes reflection as a major component of the course (Reynolds & Rice, 2006). In addition, outcomes written by influential educational bodies such as National Council of Teacher’s of English (ncte.org), The Common Core State Standards Initiative (corestandards.org), and Council of Writing Program Administrators (wpacouncil.org) emphasize the importance of metacognitive skills and demonstrate a shared belief in its importance.

But students aren’t necessarily on board. It is the rare student who has engaged in critical reflection in the academic setting. Instead, many aren’t sure how to handle it. Is it busy work from the teacher? Are they supposed to reveal their deep, inner feelings or is it a cursory overview? Is it going to be graded? What if they give a “wrong” reflection? And, according to one group of students I had, “isn’t this, like, for junior high kids?” In this last question we again see the developing learner identity. The students were essentially wondering, “does this reflective work make us little kids or grown ups?”

If we want new college students to engage in the kind of reflective work that will help them develop transferable metacognitive skills, we need to be thoughtful about how we integrate it into the coursework. Intentionality is important because there are a number of ways teachers might accidentally perpetuate these student mindsets. In order to get the most from reflective activities in class, keep the following ideas in mind:

  1. Talk openly with students about metacognition. If we want students to become aware of their learning, then the first thing to do is draw their attention to it. We should explain to students why they might care about metacognitive skills, as well as the benefits of investing themselves in the work. If we explain that reflection is one kind of metacognitive activity that helps us retrieve, sort, and choose problem-solving strategies, then reflection ceases to be “junior high” work and instead becomes a scholarly, collegiate behavior.
  2. Design very specific reflective prompts. When in doubt, err on the side of more structure. Questions like “what did you think about the writing assignment” seem like they would open the door to many responses; actually they allow students to answer without critically examining their writing or research decisions. Instead, design prompts that require students to critically consider their work. For example, “Describe one writing choice you made in this essay. What was the impact of your decision?”
  3. Integrate reflection throughout the semester. Ask students to reflect mid-way through the processes of drafting, research, and writing. If we wait until they finish an essay they learn that reflection is simply a concluding activity. If they reflect mid-process they become aware of their ability to assess and revise their strategies more than once. Also, reflection is a metacognitive habit of mind (Tarricone, 2011; Yancey, 1998) and habits only come to us through repeated activity.

These three strategies are a very basic beginning to integrating metacognitive activities into a curriculum. Not only do they help students evaluate the effectiveness of their attempts at problem solving, but they can also direct the students’ attention toward the strategies they’ve already brought to the class, thereby creating a sense of control over their learning. In the first-year writing classroom, where students are distracted and worried about life circumstance and learner identity, the sense of control gained from metacognitive work is especially important.

 

References

Chinich, M. (Producer), & Hughes, J.H. (Director). (1986). Ferris Beuller’s day off.[Motion picture]. USA: Paramount Pictures.

Reynolds, N., & Rice, R. (2006). Portfolio teaching: A guide to instructors. Boston, MA: Bedford St, Martin’s.

Tarricone, P. (2011). The taxonomy of metacognition. New York: Psychology Press.

Yancey, K.B. (1998). Reflection in the writing classroom. Logan: Utah State University Press.

(2013). First-year writing: What good does it do? Retrieved from http://www.ncte.org/library/nctefiles/resources/journals/cc/0232-nov2013/cc0232policy.pdf

(2014). Frameworks for success in postsecondary writing. Retrieved from http://wpacouncil.org/framework

(2014). English language arts standards. Retrieved from http://www.corestandards.org/ELA-Literacy/introduction/key-design-consideration/


Comprehension Monitoring: The Role of Conditional Knowledge

By Antonio Gutierrez, Georgia Southern University

In my previous post, Metacognitive Strategies: Are They Trainable?, I explored the extent to which metacognitive strategies are teachable. In my own research on how well students monitor their comprehension during learning episodes, I discovered that students reported already having a repertoire of metacognitive strategies. Yet, I have often found, in my own teaching and interaction with undergraduate and even graduate students, that having metacognitive declarative knowledge of strategies is often not sufficient to promote students’ comprehension monitoring. For instance, students may already know to draw a diagram when they are attempting to learn new concepts. However, they may not know under which circumstances it is best to apply such a strategy. When students do not know when, where and why to apply a strategy, they may in fact be needlessly expending cognitive resources for little, to no, benefit with respect to learning.

Schraw and Dennison (1994) argued that metacognition is divided in to knowledge and regulation components. Knowledge is comprised of declarative knowledge about strategies, procedural knowledge of how to apply them, and conditional knowledge about when, where, and why to apply strategies given task demands. The more that I engage students, inside and beyond my classes, the more that I become convinced that the greatest lack in metacognitive knowledge lies not in declarative or procedural knowledge, but in conditional knowledge. Students clearly have a repository of strategies and procedures to apply them. However, they seem incapable of applying those strategies effectively given the demands of the learning tasks in which they engage. So how can we enhance students’ conditional knowledge? Let’s assume that Sally is attempting to learn the concept of natural selection in her biology lesson. As Sally attempts to connect what she is learning with prior knowledge in long-term memory, she realizes she may have misconceptions regarding natural selection. She also understands that she has a variety of strategies to assist her in navigating this difficult concept. However, she does not know or understand which strategy will optimize her learning of the concept. Thus, she resorts to a trial-and-error utilization of the strategies she thinks are “best” to help her. Here we see a clear example of lack of adequate conditional knowledge. Much time and cognitive effort can be saved if we enhance students’ conditional knowledge. Calibration (the relationship between task performance and a judgment about that performance; Boekaerts & Rozendaal, 2010; Keren, 1991), a related metacognitive process, but distinct from conditional knowledge, involves the comprehension monitoring element of metacognitive regulation. As I continue my scholarship to deepen my understanding of calibration, I wonder whether conditional knowledge and calibration are more closely associated than researchers assume.

In my recent research on calibration I have often asked why the body of literature on calibration is inconclusive in its findings with respect to the effects of metacognitive strategy training on calibration. For instance, some studies have found positive effects for calibration (e.g., Gutierrez & Schraw, in press; Nietfeld & Schraw, 2002) while others have demonstrated no effect for strategy training on calibration (e.g., Bol et al., 2005; Hacker et al., 2008). This inconclusive evidence has frustrated me not only as a scholar but as a teacher as well. I suspect that these mixed findings in the literature on calibration may be due in part because researchers on calibration have neglected to address participants’ metacognitive conditional knowledge. How can we possibly hope as instructors to improve students’ comprehension monitoring when the findings on the role of metacognitive strategy instruction plays on calibration are inconclusive? So, perhaps as researchers/scholars of metacognition we are asking the wrong questions? I argue that by improving students’ metacognitive conditional knowledge, we can improve their ability to more effectively determine what they know and what they do not know about their learning (i.e., better calibrate their performance judgments to their actual performance). If students cannot effectively apply strategies given the demands of the learning episode (a conditional knowledge issue) how can we expect them to adequately monitor their comprehension (a regulation of learning issue)? Perhaps the next line of inquiry should exclusively focus on the enhancement of students’ conditional knowledge?

 

References

Boekaerts, M., & Rozendaal, J. S. (2010). Using multiple calibration measures in order to capture the complex picture of what affects students’ accuracy of feeling of confidence. Learning and Instruction, 20(4), 372-382. doi:10.1016/j.learninstruc.2009.03.002

Bol, L., Hacker, D. J., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy, and performance. The Journal of Experimental Education, 73, 269-290.

Gutierrez, A. P., & Schraw, G. (in press). Effects of strategy training and incentives on students’ performance, confidence, and calibration. The Journal of Experimental Education: Learning, Instruction, and Cognition.

Hacker, D. J., Bol, L., & Bahbahani, K. (2008). Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style. Metacognition Learning, 3, 101-121.

Keren, G. (1991). Calibration and probability judgments: Conceptual and methodological issues. Acta Psychologica, 77(2), 217- 273. http://dx.doi.org/10.1016/0001-6918(91)90036-Y

Nietfeld, J. L., & Schraw, G. (2002). The effect of knowledge and strategy explanation on monitoring accuracy. Journal of Educational Research, 95, 131-142.


Everyday Metacognition

by Craig Nelson, Indiana University

This is my first post for this group. I have two goals. I want to illustrate some ways that we can use metacognition in everyday, non-academic situations. And I want to begin my posts with some reflections that are naïve. Naïve in the sense that I have not digested the work already on the blog and have not turned to the metacognitive literature. This will help me recognize later when other material really challenges my own ideas. We know from work in other contexts that making initial ideas explicit rather than tacit greatly facilitates cognitive change. In the present context, we could say that it activates metacognitive processing tools.

As always, a concrete example will make this clearer. Crouch and her associates (2004) asked “Classroom Demonstrations: Learning Tools or Entertainment?” They found that just doing a demonstration in physics had little effect on students’ understanding. It was simply entertainment. If you had a relevant misconception before the demonstration, you would probably still have it afterwards. Telling someone what is wrong with their misconception or even having them listen to a carefully constructed lecture or read a carefully constructed text is “futile,” their ideas are unlikely to change (Arons, 1976). Crouch et al. tried an alternative method. Before presenting the demonstration they asked students to write down what they thought would happen and then discuss their predictions briefly with their neighbors (thus activating prior conceptions and some relevant cognitive and metacognitive frameworks). Crouch and her associates then presented the demonstration and asked the students to compare what happened to their own predictions and to discuss the comparison with their neighbors. This led to significant conceptual change.

One of the most powerful metacognitive tools is exactly this. Ask yourself what a speaker or article or book or demonstration in class or real life is likely to say or show. Make explicit predictions. Whenever possible write them down. Then monitor the extent to which your predictions work out. Congratulate yourself when you are right and ask why you were wrong when they don’t.

A second powerful general metacognitive tool is related. Ask yourself how you do something. Then ask yourself what are some alternative ways you might do it and how you might decide which one to use in the future. For example: What pattern do you follow when you shop in the grocery store? Do you start with produce or end with produce? What else? How else might you systematically shop? And now the tough part: What criteria might favor each of the patterns? For example: Ending with heavy things such as dog-food might minimize your pushing effort but it might also risk crushing more delicate items. I try to minimize the temptation to buy junk food and processed carbohydrates generally. This means that in the grocery stores I visit, I generally shop the margins (produce, meat, diary) and avoid going through the aisles with canned food, sweets, chips and related foods unless I have something on my list that is found there. The bottom line is that for the things we do and the ways that we think, we should remember to ask, first, what are the alternatives and, second, what do we gain and lose by the ones that we choose.

I will close with two foreshadowings of points I expect to develop much more extensively later. Learning to think is a strange enterprise. Our best thinking at each point has limits that we cannot see and may not even be able to comprehend even if someone points them out to us. Misconceptions are a basic example. It is very hard to avoid taking any contradictory evidence we encounter and distorting it so that it seems to support our initial misconception (Grant, 2009). We have to make predictions or engage in strangely structured discussions (What would it take to convince you to switch to a new view if you held this misconception? Grant, 2009) or otherwise be effectively challenged. Seek out such challenges to even your most seemingly solid ideas.

This inability to see new ways of thinking applies even to the general way we perceive reality. Suppose that you think that knowledge in general and science and math in particular are based on objective truth and is likely to be eternally true. You then might have deep trouble with the titles and core ideas in for example, Kline’s Mathematics: The Loss of Certainty, Ioannidis’ Why Most Published Research Findings Are False or Freedman’s Lies, Damned Lies, and Medical Science. Even deeper challenges would be presented by, among many others, Anderson’s Reality Isn’t What It Used To Be, or Baxter Magolda’s Authoring Your Life. But each of these implicitly or explicitly presents a metacognitive framework that can be very powerful once we master it. So my final hint today for metacognitive awareness is to play Elbow’s believing game: See if you can understand how an author comes to his or her conclusions even when they seem very different from your own. Or as Russell put it, the rationale for studying the history of philosophy is to understand how an intelligent person ever came to believe such things as a tool for recognizing the limits of one’s own beliefs. We need to do this broadly, not just historically.

 References

Anderson, Walter Truett. 1990. Reality Isn’t What It Used to Be: Theatrical Politics, Ready-To-Wear Religion, Global Myths, Primitive Chic, and Other Wonders of the Postmodern World. Harpercollins.

Arons, Arnold. Arons, A. B. 1976. Cultivating the capacity for formal operations: Objectives and procedures in an introductory physical science course. American Journal of Physics 44: 834-838.

Baxter Magolda, Marcia B. 2009. Authoring Your Life: Developing an Internal Voice to Navigate Life’s Challenges. Stylus.

Crouch, Catherine H., A. P. Fagen, P. Callan and E. Mazur. 2004. “Classroom Demonstrations: Learning Tools or Entertainment?” American Journal of Physics 72:835-838.

Elbow, Peter. 1973. Writing Without Teachers. Oxford University Press.

Freedman, David. 2010. Lies, Damned Lies, and Medical Science. Atlantic. http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/?single_page=true (or http://bit.ly/11aAmt0).

Grant, B. W. 2009. Practitioner Research Improved My Students’ Understanding Of Evolution By Natural Selection In An Introductory Biology Course. Teaching Issues and Experiments in Ecology, 6(4). http://tiee.ecoed.net/vol/v6/research/grant/abstract.html

Ioannidis, John. 2005. Why Most Published Research Findings Are False. PLoS Medicine August; 2(8): e124. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ (The most downloaded article in the history of PLoS Medicine.)

Kline, Morris. 1982. Mathematics: The Loss of Certainty. Oxford University Press.

Russell, Bertrand. 1945. A History of Western Philosophy. Simon & Schuster.


What Metacognitive Skills Do Developmental College Readers Need?

by Roman Taraban, Dmitrii Paniukov, and Michelle Kiser

Texas Tech University

In a recent post to the CASP (College Academic Support Programs) listserve, a skeptical developmental programs instructor asked why more attention can’t be given to remedial readers when designing instruction for developmental education. The instructor’s concern highlights the question:  What do we know about students who are not “college-ready” and who enroll in developmental coursework? In particular, where does metacognition fit into their development as skilled readers?

We know that reading ability, as measured by standardized instruments, like the SAT reading test for high-school students, is significantly associated with reading comprehension (Taraban, Rynearson, & Kerr, 2000). But what underlies this reading ability and can it be enhanced in college students?  Prior research revealed a several things.  As University students progress from freshman to senior years, they show small but significant growth in their use of metacognitive reading strategies (Taraban, 2011). This growth happens naturally – i.e., college students typically do not take courses that teach metacognition.  In trying to deliberately develop metacognitive reading strategies in developmental reading students, however, we found that the process can be slow and costly, but it can be done!  In a study of developmental college readers, it took roughly one semester of regular practice with a look-back reading strategy (Garner, 1987) in order to show significant improvement in reading comprehension (Taraban et al., 1997).  In addition to semester-long practice, the intervention was implemented in one-on-one tutoring, pointing to the instructional costs of bringing about detectable gains in reading skills in a remedial population.

Recently my colleagues and I had an opportunity to work with developmental readers who were enrolled in a developmental reading course at a major public research university. The students were primarily freshmen (mean number of completed credits = 16.7). We were primarily interested in three questions: 1) Could a teacher-implemented intervention improve these students’ comprehension and retention of ideas from expository texts? 2) Which metacognitive reading strategies did these students apply on their own? and 3) Was students’ use of metacognitive strategies associated with better retention of information?

The students were asked to read two expository passages and to recall as much as they could either immediately or after a 48-hour delay. They were told that they would be asked later to recall the information from the texts, but they were not prompted to apply any specific learning strategies. The two texts used for the study were each about 250 words in length and had an average Flesch-Kincaid readability score of 8.2 grade level. The passages contained 30 idea units each. Idea units are simple units of meaning derived from the text, and here were used to score the recall data. The participants read and studied one of the passages without interruption (Uninterrupted Condition), and they read and studied one of the passages paragraph-by-paragraph, and then all together (Segmented Condition).  Participants spent an equal amount of total time (10 minutes) reading and studying each of the texts. After they recalled the information, we asked them to report the strategies they used to learn the information. The specific self-reported strategies were organized into six types, as shown in Table 1. To score the strategy-use data, participants were given credit for multiple strategy types, but not for repetitions of the same strategy for the same text.

 TABLE 1: Key Types of Self-Reported Strategies

  1. REPETITION: Re-Reading; Memorize; Repetition
  2. FOCUSING ON SPECIFIC ELEMENTS: Key words; Key concepts; Grouping terms or sentences; Identifying related concepts; Parts that stood out; Parts that were difficult
  3. SELF TESTING: Summarizing; Recalling; Quizzing self; Forming acronyms
  4. GENERATING COGNITIVE ELABORATIONS: Activating prior knowledge; Recalling related experiences; Re-explaining parts of the text in other ways; Comparing and contrasting ideas; Using analogies; Using mental imagery
  5. SEGMENTATION: Grouping sentences for purposes of study; Divide by paragraph
  6. GENERAL: Read slowly; Read thoroughly; Concentrate; Understand passage

Regression analyses were conducted in order to evaluate the effectiveness of the reading approach (Uninterupted vs Segmented) in conjunction with participants’ self-reported use of the six strategy types (see Table 1). Turning to the immediate test, the reading approach mattered. When participants read and studied a segmented text they had significantly higher recall of idea units (M = 11.64) compared to non-segmented text (M = 7.93). Further, all of the participants reported using reading strategies.  Of the six strategy types, participants’ application of FOCUSING ON SPECIFIC ELEMENTS during reading was strongly associated with better recall of information from the text, and REPETITION was also important.  Considering the delayed test next, the reading approach used for the text that was read two days earlier did not matter.  However, using the strategy type SELF TESTING during reading was strongly associated with better recall, and FOCUSING ON SPECIFIC ELEMENTS was also helpful.

To address our skeptical developmental instructor, our data suggest that developmental reading instructors can structure how students process information in order to increase the number of ideas students retain, for follow-up activities like inferencing and brainstorming. The data also showed that developmental readers naturally use metacognitive reading strategies to boost their retention of information both immediately and at a delay.  Interestingly, there is no single best strategy.  Rather, FOCUSING ON SPECIFIC ELEMENTS during reading is most effective for immediate retention and SELF-TESTING during reading is most effective for longer-term retention.  Developmental students’ natural disposition to apply strategies may open opportunities for instructors to further guide, enhance, and channel these metacognitive skills to better benefit students.  What is heartening in these data is the finding that these academically-challenged students self-initiate metacognitive activities to monitor and regulate their study behaviors in order to enhance their academic performance.

References

Garner, R. (1987). Metacognition and reading comprehension. Norword, NJ: Ablex.

Taraban, R. (2011). Information fluency growth through engineering curricula: Analysis of students’ text-processing skills and beliefs. Journal of Engineering Education, 100(2), 397-416.

Taraban, R., Becton, S., Shufeldt, M., Stirling, T., Johnson, M., & Childers, K. (1997). Developing underprepared college students’ question-answering skills. Journal of Developmental Education, 21 (1), 20-22, 24, 26, 28.

Taraban, R., Rynearson, K., & Kerr, M. (2000). College students’ academic performance and self-reports of comprehension strategy use. Journal of Reading Psychology, 21, 283-308.


Faculty Metacognition of Verbal Questioning

by Charity Peak, U.S. Air Force Academy*

Few faculty would argue that teaching requires asking questions of students, but rarely do instructors consider the what, how, or why of their verbal questioning behavior.  Without metacognition of questioning strategies, this foundational instructional technique can be wasted on habit rather than design.

Faculty question students for a variety of reasons.  Surprisingly, most faculty use verbal questioning as a classroom management technique.  This might look something like a machine gun approach, firing question after question in multiple directions in an effort to keep the class engaged.  See a student dozing? Fire!  Someone checking Facebook? Fire!  Some researchers estimate that teachers ask as many as 120 questions per hour—a question every 30 seconds (Vogler, 2005)!While this strategy may keep students on their toes, it does not necessarily aid student learning.  Often these questions are low level cognitive questions, requiring mainly recall of factual knowledge.  If teachers wish to develop deeper levels of thinking, they must stimulate their students’ own evaluation of the content rather than merely requesting regurgitation of the basics.

At the other end of the spectrum is a master teacher’s approach to instruction that utilizes a specific questioning taxonomy proven to be effective for a variety of disciplines.  Rather than using the run-and-gun approach, this faculty member masterfully leads students from one point to another through a series of thoughtfully derived questions.  He or she might start with the big picture and lead to a specific point or, in contrast, begin with minutia but guide students to one main relevant theme by the end of class.  Watching these instructors in action is often humbling.  However, even these most masterful teachers are often not cognitively aware of the strategies they are using.  They have figured out what works over time, but they likely can’t point to a specific methodology they were using to support their instruction.  Rather than shooting in the dark over many years, faculty would be wise to understand the metacognition behind verbal questioning if they wish to be effective in creating higher order thinking in their students.

Moving beyond simple recall in questioning is certainly good advice for creating more opportunities in thinking, but it’s easier said than done.  Faculty often report feeling uncomfortable trying new questioning strategies.  Asking higher order thinking questions for application, analysis, and synthesis often creates extensive dead air time in the classroom.  More difficult questions require more time to think, often in silence.  Also, students are reluctant to change the very well-established classroom culture of “getting the answer right.”  Based on years of classroom experience, students will often fire answers back, playing the game of “Guess what’s in the teacher’s head.”

Despite these cultural norms, it is possible through metacognition to improve verbal questioning.  Some scholars argue that faculty should understand some of the basic questioning taxonomies that exist and how they influence learning.  For example, asking open-ended versus closed-ended questions will alter the cognitive level of thinking and response (Rothstein & Santana, 2011).  Open-ended questions tend to achieve thinking which is higher on Bloom’s Taxonomy.  Students are required to generate thoughtful answers to questions as opposed to firing one to three word facts.  For example, instead of asking, “What is an adverb?” faculty might ask students to apply their learning by identifying an adverb in a sentence or even creating their own sentences using adverbs.  Better yet, The Right Question Institute (Rothstein & Santana, 2011) encourages faculty to get students to ask their own questions rather than teachers doing all the work.  After all, the person generating the questions is arguably the person who is learning the most.

Other scholars suggest that faculty should consider the sequencing and patterns that are possible when asking questions (Vogler, 2005).  For example, cognitive psychologists often suggest a funneling or convergent questioning technique, which leads students from big picture to details because it mirrors the cognitive functioning of the brain.  However, depending on the subject area, faculty may find success in guiding students from narrow to broad thinking (divergent) by first asking low-level, general questions followed by higher-level, specific questions.  Some disciplines lend themselves to using a circular path to force critical thinking in students.  This pattern asks a series of questions which eventually lead back to the initial position or question (e.g., “What is justice?”).  While students often find these patterns frustrating, it emphasizes to students the value of thinking rather than correctly identifying the right answer.

Ultimately, though, faculty would be wise to spend less energy on the exact strategy they plan to use and instead focus on the main goals of their questioning.  In Making Thinking Visible (Ritchhart, Church, & Morrison, 2011), the authors propose that the purpose of questioning is really to make our students’ thinking visible by understanding our own expert-level thinking—aka metacognition.   To do this, the authors suggest that instead of complex taxonomies and patterns, we should focus our efforts on three main purposes for questioning in our classes:

  1. Modeling our interest in the ideas being explored
  2. Helping students to construct understanding
  3. Facilitating the illumination of students’ own thinking to themselves (i.e., metacognition)

By asking authentic questions – that is, questions to which the teacher does not already know the answer or to which there are not predetermined answers – instructors create a classroom culture that feels intellectually engaging, fosters a community of inquiry, and allows students to see teachers as learners (31).  Faculty must frame learning as a complex communal activity rather than the process of merely accumulating information.  Thoughtful questioning creates this classroom climate of inquiry, but only if faculty are metacognitive about their purpose and approach to using this critical pedagogical strategy.  Without metacognition, faculty risk relying on the machine gun approach to questioning, wasting valuable class time on recall of factual information rather than elevating and revealing students’ thinking.

Ritchhart, R., Church, M., and Morrison, K. (2011). Making thinking visible: How to promote engagement, understanding, and independence for all learners. San Francisco: Jossey-Bass.

Rothstein, D., and Santana, L. (2011). Make just one change: Teach students to ask their own questions. Boston: Harvard Education Press.

Vogler, K. E. (2005). Improve your verbal questioning. The Clearing House, 79(2): 98-103.

 

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


Using metacognitive awareness to facilitate healthy engagement with moral issues

By John Draeger, SUNY Buffalo State

As the new semester begins, I am again looking out on a classroom full of students eager to discuss “hot button” moral issues (e.g., abortion, euthanasia, hate-speech, same-sex marriage, drug legalization). In an earlier post entitled, “Using metacognition to uncover the substructure of moral issues,” I argued that metacognitive awareness can help students move beyond media pundit drivel and towards a more careful consideration of moral issues. In “Cultivating the habit of constructive discomfort”, I argued that learning requires cultivating a certain healthy discomfort (much like the discomfort often associated with vigorous exercise) and it is metacognitive awareness that keeps us within our own “zone of proximal development” (Vygotsky 1978). This post considers some of the sources of discomfort that threaten to undermine the discussion of moral issues.

Confronting “hot button” moral issues can be difficult because each of us brings our own complicated history to the conversation (replete with hang ups and blind spots). Based on my many years of teaching moral philosophy, I offer the following list of items that I found seem to derail discussion. The list is by no means exhaustive and whether these are the elements most likely to impede engagement is ultimately an empirical question that the needs to be answered. However, I argue that all of us (instructors, students, those outside the classroom) need to be aware of our own sources of discomfort with moral matters if we hope to move beyond them and towards a healthy engagement with these important issues.

Sources of discomfort: 

(1) Entrenched beliefs— some moral issues are difficult to consider because they force us to confront our foundational values.  For example, those from a wide variety of religious traditions can find it difficult to be completely open-minded to the possibility that abortion and same-sex marriage could be permissible. While they can summarize a particular position on the issue (e.g., for a particular course assignment), many find it difficult to move beyond a “bookish” articulation of the problem towards a genuine consideration of the issues because it threatens to undermine other firmly held beliefs (e.g., religious teachings).

(2) Peer pressure — many students find it difficult to swim against the current of peer opinion. When discussing sex, for example, students want to avoid being seen as either too prudish or too perverted. Sometimes students have views that fall outside the range of perceived acceptability but they refuse to voice them for fear of social disapproval. Other times, it doesn’t even occur them to consider anything outside the norm. In both cases, peer pressure can undermine full consideration of the issues.

(3) Self-interest — shifts in moral position require changes in our behavior. For example, “buying into” arguments for animal rights might demand that we change our eating habits. Often, it is easy to discount these arguments, not because they lack merit, but because we do not want to make the lifestyle changes that might be required if we became convinced by the argument.

(4) “Afraid of looking in the mirror” — discussions of moral issues can reveal uncomfortable truths about ourselves. Discussions of racial and gender discrimination, for example, can make us uncomfortable when we realize that we (or those we love) have attitudes and behaviors are insensitive and even hurtful.

(5) Ripple effects — because moral issues are interrelated, modifying our view on one issue can send ripple effects through our entire conceptual system.  For example, a discussion of euthanasia might lead us to the conclusion the quality life is important and even that some lives are no longer worth living (e.g., extreme pain without the prospect of relief). If true, then we might come to believe that it be better if some people were never born (e.g., extreme pain without the prospect of relief). Thus, thinking carefully about euthanasia might change our view of abortion. Likewise, becoming convinced by arguments for individual freedom in one area (e.g., free speech) can lead us to rethink our views in other areas (e.g., drug legalization, abortion, hate speech). However, if a student senses that a ripple might turn into a tidal wave, they often disengage.

In each case, becoming aware of the sources for our discomfort can help us move beyond a superficial consideration of the issues. In particular, asking a series of metacognitive questions can help uncover whether the discomfort is healthy (e.g., struggle with unfamiliar or difficult material) or unhealthy (e.g., blocked by entrenched beliefs, peer pressure, self-interest, or an inability to look in the mirror).

Questions we might ask our students (or even ourselves):

  • To what extent is my thinking on particular issue being influenced by my firmly held beliefs, the views of my peers, self-interest, a reluctance to take an honest look in the mirror, or concerns about the need revise my entire ethical system?
  • Am I taking the moral issue under consideration seriously? Why or why not?
  • Would I be willing to change my stance if the argument was compelling? Why or why not?
  • Is there something about the view that I cannot bring myself to consider? If so, what?

While awareness of our various blind spots and areas of discomfort will not automatically improve the quality of discussion, it can pave the way for a more meaningful consideration of the issues. As such, metacognitive awareness can facilitate healthy engagement with moral issues.

Reference:

Vygotsky, L. S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.


Testing Improves Knowledge Monitoring

by Chris Was, Kent State University

Randy Isaacson and I have spent a great deal of time and effort creating a curriculum for an educational psychology class to encourage metacognition in preservice teachers. Randy spent a number of years developing this curriculum before I joined him in an attempt to improve the curriculum and use the curriculum to test hypotheses regarding improvement of metacognition with training for undergraduate preservice teachers. A detail description of the curriculum can be found in the National Teaching and Learning Forum (Isaacson & Wass, 2010), but I wanted to take this opportunity to give a simple overview of how we structured our courses and some of the results produced by using this curriculum to train undergraduates to be metacognitive in their studies.

With our combined 40+ years of teaching, we our quite clear that most undergraduates do not come equipped with the self-regulation skills that one would hope students would acquire before entering the university. Even more disappointing, is students lack the metacognition required to successfully regulate their own learning behaviors. Creating an environment that not only encourages, but also requires students to be metacognitive is not a simple task. However, it can be accomplished.

Variable Weight-Variable Difficulty Tests

The most important component of the course structure is creating an environment with extensive and immediate feedback. The feedback should be designed to help the student identify specific deficiencies in his or her learning strategies and metacognition.  We developed an extensive array of learning resources which guide the student to focusing on knowing what they know, and when they know it. The first resource we developed is a test format that helps the students reflect and monitor their knowledge regarding the content and items on the test. In our courses we have students judge their accuracy and confidence in their responses for each item and having them predict their scores for each exam

Throughout the duration of the semester in which they were enrolled in the course students are administered a weekly exam (the courses meet Monday, Wednesday and Friday with the exams occurring on Friday). Each examination is based on a variable weight, variable difficulty format. Each examination contained a total of 35 questions composed of 15 Level I questions that were at the knowledge level, 15 Level II questions at the evaluation level, and 5 Level III questions at the application/synthesis level. Scoring of the exam was based on a system that increased points for correct responses in relation to the increasing difficulty of the questions and confidence in responses: Students choose 10 Level I questions and put those answers on the left side of the answer sheet. These 10 Level I questions are worth 2 points each. Ten Level II questions were worth 5 points each are placed on the left side of the answer sheet, and three Level III questions were worth 6 points each are placed on the left. Students were also required to choose the questions they were least confident about and place them on the right side of the answer sheet. These questions were only worth one point (5 of the 15 Level I and II questions, and 2 of the 5 Level III questions). The scoring equaled a possible 100 points for each exam. Correlations between total score and absolute score (number correct out of 35) typically range from r = .87 to r = .94.  Although we provide students with many other resources to encourage metacognition, we feel that the left-right test format is the most powerful influence on student knowledge monitoring through the semester.

The Results

Along with our collaborators, we have conducted a number of studies using the variable weight-variable difficulty (VW-VD) tests as a treatment. Our research questions focus on whether the test format increases knowledge monitoring accuracy, individual differences in knowledge monitoring and metacognition, and psychometric issues in measuring knowledge monitoring. Below is a brief description of some of our results followed.

Hartwig, Was, Isaacson, & Dunlosky (2011) found that a simple knowledge monitoring assessment predicted both test scores and number of items correct on the VW-VD tests.

Isaacson & Was (2010) found that after a semester of VW-VD tests, knowledge monitoring accuracy on an unrelated measure of knowledge monitoring increased.


Incorporating Metacognitive Leadership Development in Class

by Lauren Scharff, U.S. Air Force Academy*

During the spring 2014 semester I decided to try an explicitly metacognitive approach to leadership development in my Foundations for Leadership Development course in the Behavioral Sciences department at the United States Air Force Academy.

I had taught the course twice before and had many discussions with other course instructors. Overall, my sense was that many of our students didn’t intentionally and systematically connect what they were doing and learning in the course with their own personal leadership development. This is despite a paper that focused on a personal leadership development plan, and a video project that focused on implementing positive change within their squadrons.

This course is an upper-level, required core courses in our curriculum, and my section was one of more than 30 with approximately a dozen different instructors teaching sections. At our institution, much of the course structure within core courses is standardized across instructors, but I had 20% of the points with which to do what I desired, as long as 10% somehow assessed accountability for lesson preparation.

I was aware of a foundation of research in skill development (e.g. Svinicki, 2004), so I knew that in order to most effectively develop skills, people need multiple opportunities for practice coupled with feedback.  Feedback leads to awareness of strengths, shortcomings, and possible alternate strategies. This understanding of skill development became intertwined with my increasing focus on metacognitive approaches. I came to the conclusion that perhaps part of the less-than-ideal student connection to the course material and objectives occurred because our course activities that were designed to support that connection didn’t provide (require) enough opportunities for practice and continued awareness, especially beyond the classroom and course requirements.

As I prepared for the semester I drew on resources from The Learning Record, which outlines Five Dimensions of Learning that connected well with goals I had for my students’ leadership development: confidence and independence, knowledge and understanding, skills and strategies, use of prior and emerging experience, and critical reflection. The site also shares well-developed activities and assignments that supported my goal of using a metacognitive approach to promote my students’ leadership development.

Ultimately, I designed my course to be centered around journal entries, which I also completed.  During each lesson we shared our understandings, questions, and reflections based on the readings, as well as examples of personal observations of leadership and our reflections on how what we were learning might be effectively applied to real situations. More specifically, the journal entries included 1) answers to guided reflection questions about each reading for each lesson, and 2) at least two personal leadership observations and analyses each week. I created a simple grading system so that I wouldn’t be overloaded with assessing journal entries every lesson. (Journal assignment)

A quick poll of my students (N=13) indicated that none of them regularly kept personal journals, and only two had ever had any sort of journal assignment for a class. Thus, this requirement for regular journal writing required a change of habit for them that also represented increased time and energy for class preparation. Although there was some adjustment, when I asked for feedback after two weeks of class, the students were unanimous in their agreement that they were more deeply reading than if I had incorporated reading quizzes for accountability and that they preferred to continue the personal and reading reflections even though they involved frequent writing. Discussion during class was deeper and more engaging than in previous semesters.

Twice during the semester, students wrote evidence-based personal development evaluations, based on a shared example from The Learning Record. Students chose examples from their journals to support their evaluations of their own leadership development. These evaluation exercises forced them to thoughtfully review their observations across the weeks of the semester and develop ongoing awareness of their leadership strengths and weaknesses as well as an understanding of alternate strategies and when/how they might be useful for their leadership efforts. (Personal Development Evaluation assignment)

I also added a question each time that had them evaluate the journal approach and course design.  We made some tweaks at mid-semester. By the end of the semester, all but one student reported that the journal entries deepened their learning and personal awareness of their leadership development. While I will likely make some further tweaks in future semesters, I believe that this approach was a success, and that it could be scaled up for larger classes (see the simplified grading scheme in the Journal assignment). Below are sample comments from the final evaluation assignment (released with student permission):

“The leadership journal has had a tremendous effect on my personal development as a leader.  The journal has made me aware of my strengths and weaknesses…. The journal allowed me an avenue to give time and actually think about how I am doing as a leader and peer within my environment.”

 

“The personal observations were definitely helpful for documenting our successes and failures, which we can look back upon and improve. This relates not only to our personal leadership development, but to how we learn about ourselves.”

 

“These journals have taken all of us on a journey through the semester.  They undoubtedly began as something that we disliked and looked down upon each week.  However, I have really grown to love and understand this application of leadership growth.  They not only provide a chance for us to look back on our leadership gains and failures, they offer an opportunity for us to challenge ourselves in order to write about the things we want to see in ourselves.  The journals have become much more than a simple task of writing on a week-to-week basis.  They have grown into an outpouring of our character and lives as we turn the page from underclassmen to upper-class leaders and eventually to lieutenants in a few short months.  I believe that these journals are also a metaphor for many leadership challenges in that they will be frequent, difficult, and time consuming, but in the end they will let us all grow.  ….my reflections are not simply babble, …they actually represent significant growth and understanding of myself.”

References:

Svinicki, Marilla. 2004. Learning and Motivation in the Postsecondary Classroom. Bolton, MA: Anker Publishing Co, Inc.

The Learning Record: http://www.learningrecord.org/

————————————-

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.

 


Metacognition and Reflective Thinking

By Steven C. Fleisher, California State University Channel Islands

Imagine that we are reading an assignment. As we read, do we think: “How long will this take?” “Will this be on the test?” If so, try this instead. Presume that we are reading the article as preparation for meeting later with an important person such as our supervisor to discuss the article. How would this situation change the questions we ask ourselves? Such thinking can make us aware of what constitutes satisfactory mastery of knowing and how to achieve it.

Think back for a moment to learning a psychomotor skill, such as learning to ride a bicycle. It is normal to master that skill with normal innate balance and strength. We might think: “That’s all there is to it.” However, watching cyclists in a serious bicycle race or triathlon, reveals that reliance only on innate ability cannot produce that kind of performance. That level of expertise requires learning to pedal with cadence, to deliver equal power from both legs, use the gearing appropriately, exploit position within a group of racers and pace oneself relative to challenges. Untrained innate ability can rarely get us far in comparison to the results of informed training.

The same is true in learning. Metacognitive skills (learnable skills) enhance academic performance. People with metacognitive skill will usually outperform others who lack such skill, even others with greater innate intelligence (natural ability). Metacognitive training requires developing three strengths: 1) metacognitive knowledge, 2) metacognitive monitoring, and 3) metacognitive control.

Metacognitive knowledge refers to our understanding about how learning operates and how to improve our learning. We should have enough of this knowledge to articulate how we learn best. For example, we can know when it is best for us to write a reflection about a reading in order to enhance our learning. We should be alert to our misconceptions about how our learning works. When we learn that cramming is not always the best way to study (Believe it!), we must give that up and operate with a better proven practice.

Metacognitive monitoring refers to developed ability to monitor our progress and achievement accurately. For example, self-assessment is a kind of metacognitive monitoring. We should know when we truly understand what we are reading and assess if we are making progress toward solving a problem. When we become accurate and proficient in self-assessment, we are much better informed. We can see when we have mastered certain material well enough, and when we have not.

Metacognitive control. This competency involves having the discipline and control needed to make the best decisions in our own interests. This aspect of metacognition includes acting on changing our efforts or learning strategies, or taking action to recruit help when indicated.

Putting it together. When we engage in metacognitive reflection, we can ask ourselves, for example, “What did we just learn?” “What was problematic, and why?” “What was easy, and why?” “How can we apply what we just learned?” Further, when we gain metacognitive skill, we begin to internalize habits of learning that better establish and stabilize beneficial neural connections.

Reflective Exercises for Students:

  1. Metacognitive knowledge. Consider three learning challenges: acquiring knowledge, acquiring a skill, or making an evidence-based decision. How might the approaches needed to succeed in each of these three separate challenges differ?
  2. Metacognitive monitoring. After you complete your next assignment or project, rate your resultant state of mastery on the following scale of three points: 0 = I have no confidence that I made any meaningful progress toward mastery; 1 = I clearly perceived some gain of mastery, but I need to get farther; 2 = I am currently highly confident that I understand and can meet this challenge.
  3. Next, see if your self-rating causes you to take action such as to re-study the material or to seek help from a peer or an instructor in order to achieve more competence and higher confidence. A critical test will be whether your awareness from monitoring was able to trigger your taking action. Another will come in time. It will be whether your self-assessment proved accurate.
  4. Metacognitive control. To develop better understanding of this, recall an example from life when you made a poor decision that proved to produce a result that you did not desire or that was not in your interests. How did living this experience equip you to better deal with a similar or related life challenge?

References

Chew, S. L. (2010). Improving classroom performance by challenging student misconceptions about learning. Association for Psychological Science: Observer, Vol. 23, No. 4. http://psychologicalscience.org/observer/getArticle.cfm?id=2666

Dunlosky, J. and Metcalf, J. (2009). Metacognition. Thousand Oaks, CA: Sage.

Leamnson, R. N. (1999). Thinking about teaching and learning: Developing habits of learning with first year college and university students. Sterling, VA: Stylus.

Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41(4), 219-225.

Wirth, K. (2010). The role of metacognition in teaching Geoscience. Science Education Resource Center, Macalester College. http://serc.carleton.edu/NAGTWorkshops/metacognition/activities/27560.html


Waves of Insight about Teaching and Learning

James Rhem, Executive Editor, The National Teaching & Learning FORUM

“When I began The National Teaching & Learning FORUM” over twenty years ago, almost everyone in faculty development had heard of Bloom’s Taxonomy. I hadn’t since I was new to this beat; so I went to the University of Wisconsin’s Memorial Library and looked for it. There on the shelf was what they were talking about, the taxonomy of cognition, but right beside it was something that interested me as much if not more, the taxonomy of affect. No one (or at least no one I knew) had heard of this taxonomy or the one on the psychomotor domain that followed the one on affect. I began quietly beating the drum for an awareness of affect back in 1995 (according to a search of my old email). That was the same year Daniel Goleman’s “Emotional Intelligence” came out; so I suspected I’d stumbled onto what would become a cresting wave. That wave has grown, but it’s still heading toward a crest.

I go into all this because I think I can already see the waves of insight and meaningful rethinking of teaching and learning that will follow. Metacognition as a concept has been around for a long time, but coming to understand it and its place in learning (and thus is effective teaching) is a new thing. It’s something I’ve been increasingly covering in the FORUM and something I plan on continuing to cover. Indeed, NTLF created a series of small books with Stylus a few years back and I would be remiss if I didn’t mention that one of the titles in that series is about metacognition (Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy Edited by Matthew Kaplan ,  Naomi Silver ,  Danielle LaVaque-Manty ,  Deborah Meizlish Foreword by James Rhem   https://sty.presswarehouse.com/Books/BookDetail.aspx?productID=298776 ) 

I’ve linked two recent articles from the FORUM on metacognition. And I invite readers of this list who may have written (or plan to write) articles on metacognition and teaching and learning to submit material to me for wider distribution.  

1. Metacognitive Skills – why bother (and how)? by Carol Hostetter and Leah Savion at Indiana University

2. Metacognition and Disciplinary Thinking by Matt Fisher at Saint Vincent College

So what’s the next wave after metacognition?  The psychomotor domain. Count on it. 

James Rhem, Executive Editor, The National Teaching & Learning FORUM


Meta-Teaching: Improve Your Teaching While Improving Students’ Metacognition

By Aaron S. Richmond, Metropolitan State University of Denver

To-date, many of the wonderful blogs posted on Improve with Metacognition  have aptly focused on metacognition itself. They varied from classroom exercises to improve student metacognition (see Westmoreland, 2014) to increasing higher level thinking through metacognitive practices (see Nuhfer, 2014) to the merits of measuring metacognition (Was, 2014). Not yet covered is that of the secondary purpose of this website. That is, the emphasis will be the process of teaching about metacognition and teaching metacognitively. As such, there is great potential and room discourse on meta-teaching or meta-pedagogy as a way to both improve student learning of metacognition and our own teaching practices.

Yet first, it is important to begin with a solid operational definition of meta-teaching. As Chen (2013) states,

Like meta-cognition and meta-learning, meta-teaching, as ‘teaching about teaching’, can serve to design, examine and reflect on teaching. From practice-orientation, it defines what teaching activity is and what it is for, under which theoretical framework it is being carried out, and what experience and rules can be applied to it. Meanwhile, meta-teaching can assist teachers in discovering drawbacks in the teaching system and solving problems. This demonstrates that meta-teaching contains such functions such as understanding teaching, changing teaching and reflecting on teaching. (p. S64)

Therefore, by using this definition, how can we first improve our teaching using meta-teaching practices? And second, how can we use meta-teaching to specifically improve our teaching of metacognitive theory and the metacognition of our students?

Why Engage in Meta-Teaching Practices?

Drawn from the literature on meta-teaching, there are several benefits and reasons why college professors should employ meta-teaching practices. First, and foremost it promotes student learning (Chen, 2013). When teachers reflect and evaluate whether their teaching methods actually have an impact on student learning and adjust their practices accordingly, inevitably student learning and performance improves. Second, meta-teaching can invigorate and create a passion for teaching. In that, engaging in this process has been found to increase teacher’s love for the profession (Chen, 2013). Moreover, Chen states, “When the teacher takes action, he/she begins to observe and reflect on the action, impelling him/her to stay highly conscious of what he/she is doing….Without meta-teaching action, a teacher would hardly keep his /her motivation and enthusiasm for better teaching.” (p. S69) Finally, meta-teaching promotes the teaching profession through formal and informal scholarship of teaching and learning (SoTL). That is, SoTL cannot be conducted without proper meta-teaching practices.

Meta-Teaching Strategies Used to Improve Metacognition

Plan, strategy, monitor, and evaluate.  Spring (1985) suggests that teachers should use meta-teaching strategies by properly planning lessons, critically reflecting on appropriate instructional strategies/methods to obtain instructional goals, monitor student learning, and evaluate the efficacy of the strategy. For example, in a recent study I conducted (Richmond, in press), I used meta-teaching strategies to improve pre-service education students’ knowledge of various learning strategies. See Table 1 for an illustrative example of how I employed the meta-teaching strategies of planning, instructional strategy, student monitoring, and evaluation to improve my teaching and increase my student’s knowledge of metacognitive theory (e.g., learning strategies).

Table 1. Example of Implementing Meta-Teaching to Teach Learning Strategies
Meta-Teaching Strategy Richmond’s (in press) Educational PsychologyClassroom Example
Planning and Goals
  • Increase both higher and lower level learning of the learning strategies of rehearsal, organization, elaboration, spacing vs. massed practice, and distributed practice.
  • Increase retention of the knowledge of learning strategies.
  • Attempt to assist students in transferring these strategies to their own learning.

 

Instructional Strategy
  • Used Active learning (e.g., small group discussion, experimentation, elaboration) vs. direct instruction (e.g., lecture) to teach about learning strategies.
Monitoring Student Learning
  • Formally assessed prior knowledge of student’s understanding of learning strategies.
  • Formally assessed immediate retention of knowledge of learning strategies.
  • Formally assessed long-term retention (4-weeks) of knowledge of learning strategies.
  • Informally assessed (e.g., why questions and a 1-minute written assessment) progress of learning about learning strategies during class.
Evaluation
  • Analyzed which instructional method was most effective and found that only active learning instruction increased higher-level learning.
  • Found that students were unable to transfer strategies.
  • Found that students taught with active learning instruction retained more information over time.

Encourage metacognition in students. Not only can teachers use meta-teaching strategies to improve their own teaching, student learning, and to teach about metacognition, they can also model and encourage metacognitive thinking in their students. Similar to that of meta-teaching strategies, Chen suggests students should set a plan/goal for learning, develop a strategy to achieve this goal, monitor the progress of the given strategy, and finally evaluate the effectiveness of the strategy. Moreover, when teachers effectively use meta-teaching strategies they explain the reasoning behind specific pedagogical practices (e.g., formal and informal assessments, specific teaching strategies, assignments, etc.) and they explain the successes and failures of meta-teaching strategies (Chen, 2013). This process may model metacognitive practices to students. Additionally, Chen argues that teachers using meta-teaching strategies (no matter the content domain) always specifically infuse and teach metacognitive strategies to students.

Closing Meta-Remarks

I believe that many exemplar teachers inherently use meta-teaching strategies. However, for the rest of us, it is extremely important to investigate and learn how to improve our teaching through these practices. Additionally, when we convey the process to our students we are modeling metacognitive processes to students, which they too can use to improve their learning and performance in whatever endeavor they so choose.

References

Chen, X. (2013). Meta-teaching: Meaning and Strategy. Africa Education Review, 10(1), S63-S74. doi:10.1080/18146627.2013.855431

Nuhfer, E. (2104). Metacognition for guiding students to awareness of higher-level thinking (part 2). Retrieved from https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-2/

Richmond, A. S. (in press). Teaching learning strategies to pre-service educators: Practice what we preach! In M. C. Smith, & N. DeFrates-Densch (Eds.). Challenges and innovations in educational psychology teaching and learning. Hersey, PA: IGI Global.

Spring, H. T. (1985). Teacher decision making: A metacognitive approach. The Reading Teacher, 290-295.

Was, C. (2014). Are current metacognition measures missing the target? Retrieved from https://www.improvewithmetacognition.com/are-current-metacognition-measures-missing-the-target/

Westmoreland, D. (2014). Science and social controversy – A classroom exercise in metacognition. Retrieved from https://www.improvewithmetacognition.com/science-and-social-controversy-a-classroom-exercise-in-metacognition/


Metacognition for Guiding Students to Awareness of Higher-level Thinking (Part 2)

by Ed Nuhfer (Contact: enuhfer@earthlink.net; 208-241-5029)

Part 1 introduced the nature of adult intellectual development in terms of the stages ascended as one becomes educated. Each stage imparts new abilities that are valuable. This Part 2 reveals why awareness of these stages is important and offers metacognitive exercises through which students can begin to engage with what should be happening to them as they become better thinkers.

 

A disturbingly tiny contingent of professors in disciplines outside adult education have read the adult developmental research and recognized the importance of Perry’s discovery. Even fewer pass on this awareness directly to their students. Thus, recognition that the main value of a university education does not lie in acquired knowledge of facts and formulae but rather in acquiring higher level thinking abilities remains off the radars of most students.

Given what we know from this research, a potential exists for American higher education’s evolving into a class-based higher educational system, with a few institutions for the privileged supporting curricula that emphasize developing the advanced thinking needed for management and leadership, and a larger group of institutions fielding curricula emphasizing only content and skills for producing graduates destined to be managed. Until students in general (and parents) recognize how these two educational models differ in what they offer in value and advantages for life, they will fail to demand to be taught higher-order thinking. Overcoming this particular kind of ignorance is a struggle that neither individual students nor a free nation can afford to lose.

 

Teaching Metacognition: Mentoring Students to Higher Levels of Thinking

One way to win this struggle is to bring explicit awareness of what constitutes becoming well educated directly to students, particularly those not enrolled in elite, selective schools. All students should know what is happening to them, which requires understanding the stages of adult intellectual development and the sequence in which they occur offers the explicit framework needed to guide students to do beneficial “thinking about thinking.” (See Part 1, Table 1.) This research-based framework offers the foundation required for understanding the value of higher-level thinking. It offers a map for the journey on which one procures specific abilities by mastering successively higher stages of adult thinking. Through learning to use this framework metacognitively, individuals can start to discover their current stage of intellectual development and determine what they need for achieving the next higher stage.

I have included two exercises for students to show how the research that informs what we should be “thinking about” can be converted into metacognitive components of lessons. These modules have been pilot tested on  students in introductory general education and critical thinking courses.

The first, “Module 12 – Events a Learner Can Expect to Experience,” uses the research that defines the Perry stages (Table 1) as a basis for authoring an exercise that guides students through key points to “think about” as they start to reflect upon their own thinking. Instructors can employ the module as an assignment or an in-class exercise, and should modify it as desired. For many students, this will serve as their first exposure to metacognition. If this is the reader’s first introduction to adult intellectual development, work through this module, ideally with a colleague on a lunch break. Start to procure some of the key resources listed in the references for your personal library.

With the exception of Perry Stages 7, 8, and 9, Module 12 largely addresses the cognitive realm. However, when intellectual development occurs successfully, affective or emotional development occurs in parallel as one advances through higher cognitive stages (see Nuhfer, 2008). Metacognition or “thinking about thinking” should extend also to a reflective “thinking about feelings.” Since the 1990s, we have learned that our feelings about our learning–our affective component of thinking– influence how well we can learn. Further, our affective development or “emotional intelligence” determines how well that we can work with others by connecting with the through their feelings, which is a huge determinant in work and life success.

The second “Module 4—Enlisting the Affective Domain” helps students to recognize why the feelings and emotions that occur as one transitions into higher stages are important to consider and to understand. At the higher levels of development, one may even aspire to deeply understand another by acquiring the capacity for experiencing another’s feelings (Carper, 1978; Belenky and others, 1986).

Frequent inclusion of metacognitive components in our assignments is essential for providing students with the practice needed for achieving better thinking. Guiding students in what to “think about” can help students engage in challenges that arise at the finer scales of metadisciplines, disciplines, courses, and lessons. This requires us to go beyond articulating: “What should students learn and how can we assess this?”  by extending our planning to specify “What is essential that students should think about, and how can we mentor them into such thinking?”

REFERENCES CITED (additional references are provided in the two exercises furnished)

Arum, R. and Roksa, J. (2011). Academically Adrift: Limited Learning on College Campuses. Chicago, IL: University of Chicago Press.

Belenky, M.F., B.M. Clinchy, N.R. Goldberger, and J.M. Tarule. (1986) Women’s Ways of Knowing: The Development of Self, Voice, and Mind, New York: Basic Books. (Reprinted in 1997).

Carper, B. A. (1978). Fundamental patterns of knowing in nursing. Advances in Nursing Science 1 1 13–24.

Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–235). Hillsdale, NJ: Erlbaum.

Journal of Adult Development (2004). Special volume of nine papers on the Perry legacy of cognitive development. Journal of Adult Development (11, 2) 59-161 Germantown NY: Periodicals Service Co.

Nuhfer, E. B (2008) The feeling of learning: Intellectual development and the affective domain: Educating in fractal patterns XXVI. National Teaching and Learning Forum, 18 (1) 7-11.

Perry, W. G., Jr. (1999). Forms of intellectual and Ethical Development in the College Years. (Reprint of the original 1968 1st edition with introduction by L. Knefelkamp). San Francisco: Jossey-Bass.


Metacognition for Guiding Students to Awareness of Higher-level Thinking (Part 1)

by Ed Nuhfer (Contact: enuhfer@earthlink.net; 208-241-5029)

When those unfamiliar with “metacognition” first learn the term, they usually hear: “Metacognition is thinking about thinking.” This is a condensation of John Flavell’s (1976) definition: “Metacognition refers to one’s knowledge concerning one’s own cognitive processes or anything related to them…” Flavell’s definition reveals that students cannot engage in metacognition until they first possess a particular kind of knowledge. This reminds us that students do not innately understand what they need to be “thinking about” in the process of “thinking about thinking.” They need explicit guidance.

When students learn in most courses, they engage in a three-component effort toward achieving an education: (1) gaining content knowledge, (2) developing skills (which are usually specific to a discipline), and (3) gaining deeper understanding of the kinds of thinking or reasoning required for mastery of the challenges at hand. The American higher educational system generally does best at helping students achieve the first two. Many students have yet to even realize how these components differ, and few ever receive any instruction on mastering Component 3. Recently, Arum and Roksa (2011) summarized the effectiveness of American undergraduate education in developing students’ capacity for thinking. The record proved dismal and revealed that allowing the first two components to push aside the third produces serious consequences.

This imbalance has persisted for decades. Students often believe that education is primarily about gaining content knowledge—that the major distinction between freshmen and seniors is “Seniors know more facts.” Those who never get past this view will likely acquire a degree without acquiring any significantly increased ability to reason.

We faculty are also products of this imbalanced system, so it is not too surprising to hear so many of us embracing “covering the material” as a primary concern when planning our courses. Truth be told, many of us have so long taught to content and to skills necessary for working within the disciplines that we are less practiced in guiding our students to be reflective on how to improve their thinking. Adding metacognitive components to our assignments and lessons can provide the explicit guidance that students need. However, authoring these components will take many of us into new territory, and we should expect our first efforts to be awkward compared to what we will be authoring after a year of practice. Yet, doing such work and seeing students grow because of our efforts is exciting and very worthwhile. Now is the time to start.

Opportunities for developing metacognitive reflection exist at scales ranging from single-lesson assignments to large-scale considerations. In my first blog for this site, I chose to start with the large-scale considerations of what constitutes development of higher-level thinking skills.

 

What Research Reveals about Adult Thinking

More than five decades have passed since William Perry distinguished nine stages of thinking that successful adult intellectual development (Table 1) produces. The validity of his developmental model in general seems firmly established (Journal of Adult Development, 2004). Contained within this model is the story of how effective higher education improves students’ abilities to think and respond to challenges. Knowing this story enables us to be explicit in getting students aware of what ought to be happening to them if higher education is actually increasing their capacity for thinking. This research enables us to guide students in what to look for as they engage in the metacognition of understanding their own intellectual development.

Enhanced capacity to think develops over spans of several years. Small but important changes produced at the scale of single quarter or semester-long courses are normally imperceptible to students and instructors alike. Even the researchers who discovered the developmental stages passed through them as students, without realizing the nature of the changes that they were undergoing. For learning that occurs in the shorter period of a college course, it is easier to document measurable changes in learning of disciplinary content and the acquisition of specific skills than it is to assess changes in thinking. Research based on longitudinal studies of interviews with students as they changed over several years finally revealed the nature of these subtle changes and the sequence in which they occur (Table 1).

 

Table 1: A Summary of Perry’s Stages of Adult Intellectual Development

Stage 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers.
Stage 3 thinkers realize that authority is fallible and does not have good answers for all questions. Thinkers at this stage respond by concluding that all opinions are equally valid and that arguments are just about proponents’ thinking differently. Evidence to the contrary does not change this response.
Stage 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments.
Stage 5 thinkers can use evidence. They also accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs.
Stage 6 thinkers appreciate ambiguity as a legitimate quality of many issues. They can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon both context and value systems.
Stages 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions.

In part 2 of this blog, we will provide metacognitive class exercises that help students to understand what occurs during intellectual development and why they must strive for more than learning content when gaining an education.


Using metacognition to uncover the substructure of moral issues

By John Draeger, SUNY Buffalo State

As a moral philosopher, my introductory courses revolve around various controversial issues (e.g., abortion, euthanasia, hate speech, same sex marriage, invasions of privacy in the name of national security or commerce). It is not hard to generate discussion about these topics, but important philosophical issues often to get lost in the mayhem. My students try to keep things straight by focusing on particular bits of content. They hope that a laundry list of terms and distinctions will help them make sense of particular ethical issues. For my part, however, most of the interesting stuff occurs behind the scenes. I don’t much care about which topics we discuss because at some level I don’t think that we are talking about the particular topical issues anyway. Details matter, of course, but I am most interested in helping students uncover the underlying value conflicts common to many ethical debates. This, I argue, requires developing metacognitive awareness.

Consider three possible positions on hate speech: (1) ban hate speech on college campuses because it harms individual students, (2) allow hate speech because banning it would violate the rights of individual students, (3) allow hate speech because banning it would do more harm than good in the long-run. Now consider three possible views on governmental surveillance in the name of national security: (1) allow governmental surveillance because it promotes an important good (e.g., national security), (2) ban governmental surveillance because it violates the rights of citizens (e.g., privacy), (3) ban governmental surveillance because it does more harm than good in the long-run. Note the similar underlying value structures of these positions. One favors well-being (e.g., protect individual students or a nation) over other considerations. Another favors rights (e.g., free-speech or privacy) over harms to well-being caused by the exercise of those rights. The last considers two forms of well-being (e.g., short-term and long-term).

As an instructor, I know that teaching students a process by which they can uncover underlying value structures requires scaffolding and plenty of opportunities to practice (Duron, Limback, & Waugh, 2006). Among my many activities and assignments, I ask students to answer the following questions about each of the readings: (1) what is the author’s core insight/thesis? (2) what are the core values at issue? (3) what are the central philosophical problems at issue? (4) what are the central topics at issue? It is not long before students understand that the last two questions are not actually redundant (e.g., well-being versus rights is not the same as hate speech versus governmental surveillance).

This exercise helps students focus on what I take to be most important, namely the underlying value structure. It also sets up the next exercise in which I ask students to use the resources found in one reading (e.g., hate speech) to answer the topical question raised in another (e.g., government surveillance).  This can be difficult until students recognize there are values common across different topical debates and they recognize the similarities in the philosophical substructure (e.g., well-being over rights, rights over well-being, long-term well-being over short-term well-being). Because it isn’t always easy to fit one view into the structure of another, this exercise leads to many questions about each of the readings. As the semester moves along, we discuss each reading in relation to those that came before.  By the end of the semester, we pick author names “at random” and discuss the connections between them.

With an understanding the underlying value structure of a particular moral issue, students can begin to “think like a philosopher.” It puts them in a position to move beyond mere coffee shop conversation and the rehash of media pundit drivel towards a more careful consideration of the issues. Through the process outlined above, they begin to notice when their discussions lapse into media drivel and thus when they need steer the conversation back towards the underlying value structure. Insofar as this exercise moves students towards the ability to consciously and explicitly understand the substructure of values underlying a wide variety of ethical issues, it moves them towards a more sophisticated understanding of those issues and towards a metacognitive awareness of their own learning.

References

Duron, R., Limbach, B., & Waugh, W. (2006). “Critical Thinking Framework for Any Discipline.” International Journal of Teaching and Learning in Higher Education, 17 (2), 160-166.

 


Breaking through the barriers to learning

by David Westmoreland, U.S. Air Force Academy*

Teachers of science increasingly find themselves entangled in social controversies. This is true for physicists teaching about the origin of matter, geologists discussing the age of the planet, biologists teaching evolution, and climatologists teaching about global warming. In most cases, the science is relatively clear, and there is little controversy within the scientific community.

In my field of biology, for example, a significant percentage of students enter the classroom with preconceived notions about the theory of evolution. Public perception of this field has changed little in the past 30 years. In the most recent poll, 42% of Americans rejected evolution outright, a figure that has fluctuated between 40 and 47% for more than 3 decades (Newport 2014). Among college-educated Americans, there is greater acceptance of evolution as valid science; in 2014, about 25% espoused a creationist perspective. Still, it is surprising that one-fourth of college-educated Americans reject evolution, given the expansive effort to incorporate evolutionary biology into public education and the positive presentation of evolution in the media.

Why? One possibility is that we have we have failed to use the right approach to overcome a socially derived obstacle to learning. In undergraduate biology courses, the concept of evolution is often introduced in concert with empirical evidence supporting it, with the expectation that students will be open to ideas that, in fact, they are resistant to learn.
My research on cadets at the Air Force Academy indicates a barrier to learning evolution by about 25% of students. I sampled 147 cadet volunteers who self-categorized as creationists (rejecting evolution), theistic evolutionists (acknowledging evolutionary change with some degree of divine influence) or atheistic evolutionists (acknowledging evolution by natural processes alone). Each student responded to surveys that quantified (a) their knowledge of the subject, and (b) their perspective on evolution as science, in addition to demographic information. The results are intriguing.

Knowledge Test Score

              Creationists         36               3.7A ± 0.38
Theistic evolutionists 75 6.2B ± 0.25
Atheistic evolutionists 36 6.3B ± 0.37

Despite having a similar educational background to the other groups, creationists’ knowledge scores were roughly half those of the other groups. The difference is statistically significant.

One might think that, if creationist students were more open to learning evolutionary concepts, their acceptance of evolution might rise. But think again – the correlation of knowledge and perception is not so clear. Knowledge is significantly related to acceptance for theistic and atheistic evolutionists, but not for creationists. For them, learning facts does not appear to influence perception.

This is where metacognition comes in. In a review of 26 research articles on undergraduates’ knowledge and acceptance of evolution, Lloyd-Strovas and Bernal (2012) concluded that acceptance is related to student understanding of the nature of science – that is, science as a cultural and intellectual endeavor. When students learn that science should not be regarded as a repository of absolute truth, but rather, an ongoing effort to understand and explain the natural world, the barrier to learning is breached. As emphasized by Lombrozo et al. (2008): “…Students may be more likely to accept evolution if they understand that a scientific theory is provisional but reliable, that scientists employ diverse methods for testing scientific claims, and that relating data to theory can require inference and interpretation.”

In other words, instructors must prepare the field before engaging students in social controversies. Otherwise, students are more likely to engage in social cognition – the tendency to form opinions on the basis of social identity (Bloom and Weisberg 2007). If an individual strongly identifies herself as belonging to a group that holds a common opinion on a topic, she is likely to express that opinion even in the absence of competent understanding of the subject. For such persons, empirical information is likely to be ignored due to a fundamental desire to reinforce a social network. Consider, for example, the strong relationship between political affiliation and skepticism about global warming.

College courses are no strangers to controversy. We often engage students in debate, and have them present and defend positions. What is missing, I think, is pushing our students to critically evaluate the processes they used to form the opinions in the first place.

References

Bloom, P., and D. S. Weisberg. (2007). Childhood origins of adult resistance to science. Science 316: 996-997.

Lloyd-Strovas, J. D., and X. E. Bernal. (2012). A review of undergraduate evolution education in U.S. universities: building a unifying framework. Evolution Education Outreach 5: 453–465.

Lombrozo, T., A. Thanukos, and M. Weisberg. (2008). The importance of understanding the nature of science for accepting evolution. Evolution Education Outreach 1: 290-298.

Newport, F. (2014, June 2). In U.S., 42% believe creationist view of human origins. Retrieved from http://www.gallup.com

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


Creating a Metacognitive Movement for Faculty

by Charity Peak, U.S. Air Force Academy*

Faculty often complain that students don’t complete reading assignments.  When students do read, faculty yearn for deeper analysis but can’t seem to get it.  With SAT reading scores reaching a four-decade low (Layton & Brown, 2012) and nearly forty percent of postsecondary learners taking remedial coursework (Bettinger & Long, 2009), it’s not surprising that college students are increasingly unable to meet the reading expectations of professors.  Faculty sense the waning reading abilities of their students, but they struggle to identify how to address the problem.  After all, they weren’t trained to be reading teachers.

In February 2012, a group of faculty gathered for a Scholarship of Teaching and Learning (SoTL) Circle at the U.S. Air Force Academy to discuss how to get students to read more critically.  The topic spurred such great interest that an interdisciplinary faculty learning community on Reading Critically was formed to investigate the issue and share strategies to use in the classroom.  What evolved was a collective movement by faculty to become metacognitively aware of why and how they were assigning and apprenticing students to read more critically within their disciplines.

Our first meeting tackled the big question, “What do we want to know about college reading?”  Despite our interdisciplinary nature, we easily identified several common areas of concern:  Compliance (completing reading assignments), Comprehension (understanding what they read), and Critical Analysis.  These Three C’s of College Reading guided our discussions over the next two academic years and eventually led to the creation of a website to assist other faculty members struggling with the same issues.

As academics, our first inclination was to dive into the literature to determine what other institutions had discovered about this issue.  Surely we weren’t the only faculty grappling with these concerns. Not surprisingly, the research literature confirmed that the vast majority of college students do not read assignments ahead of time and do not consider the textbook to be a critical component of learning (Berry et al., 2010).  In fact, a number of studies find that college students only read textbooks about six hours per week (Spinosa et al., 2008), with just 20-30% reading compliance for any given day and assignment (Hobson, 2004).  Faculty hoping to set the stage prior to class and engage learners in meaningful discussions during class must first address reading compliance among students.

Unfortunately, reading is not indicative of comprehension.  The combination of students’ weak reading abilities (particularly marginalized students) and difficult textbook structure produce unskilled learners, which faculty are unprepared to handle.  Hobson (2004) explains that most college teachers – content specialists – do not realize their students are struggling to comprehend assigned texts.  Furthermore, if faculty insist on emphasizing reading as part of their course structure, then “helping students improve their reading skills should be the responsibility of every college-level teacher” (p. 4). Without specific strategies to address the reading needs of students, typically far outside the spectrum of the usual subject area specialist, faculty are rendered helpless in creating deep thinking environments in the classroom.

Because low reading compliance predicts nonparticipation (Burchfield & Sappington, 2000), college faculty must address the issue in an effort to drive deeper learning.  Over the course of two years, our Reading Critically faculty learning community identified and shared several research-based strategies to assist faculty in improving reading compliance, comprehension, and critical analysis.  With no budget and nothing more than a dedication to the cause, we invited speakers to our meetings from our own institution to share how they were apprenticing readers within their courses. We discovered the value of pre-class reading guides, concept mapping, equation dictionaries, and even reading aloud in class. The interdisciplinary connectedness and learning through a common academic concern became a welcome respite from the typical silos that exist in higher education.

By the end of our first year together, our faculty learning community had gathered a wealth of research-based practices that could be implemented in courses across all disciplines.  While each of the group’s participants had learned a great deal, we weren’t sure how to spread the word and continue the movement.  Then, we discovered Carnegie Mellon’s Solve a Teaching Problem website.  Alas, a model for us to follow!  We set out to design a website for faculty to Solve a Reading Problem.   Collaboratively, we created a step-by-step way for faculty to address reading issues they were encountering in their courses:

Step 1: Identify a reading problem

Step 2: Investigate a reason for the problem

Step 3: Initiate a strategy to address the problem

Our learning community pooled resources together by suggesting various problems and solutions along with research-based literature to support our ideas.  Faculty then submitted lesson ideas and classroom strategies they found successful in their own courses to support better reading compliance, comprehension, and critical analysis.  While the website is still very much a work in progress, it represents two years of metacognition around why faculty assign readings and how to maximize those opportunities in the classroom.

Ultimately, our faculty learned that we have a responsibility to be metacognitive about our own teaching practices in order to improve learning.  This group’s commitment to the cause created an interdisciplinary metacognitive movement among our faculty that is still developing.  What metacognitive movement can you lead at your institution?

References:

Berry, T., Cook, L., Hill, N,. & Stevens, K. (2010). An exploratory analysis of textbook usage and study habits: Misperceptions and barriers to success. College Teaching, 59(1), 31-39.

Bettinger, E., & Long, B. (2009). Addressing the needs of underprepared college students: Does college remediation work? Journal of Human Resources, 44(3), 736-771.

Burchfield, C. M., & Sappinton, J. (2000). Compliance with required reading assignments. Teaching of Psychology, 27(1), 58-60.

Hobson, E. H. (2004). Getting students to read: Fourteen tips. IDEA Paper No. 40. Manhattan, KS: The IDEA Center.

Layton, L., & Brown, E. (September 24, 2012). SAT reading scores hit a four-decade low. Washington Post. Washington, D.C.

Spinosa, H., Sharkness, J., Pryor, J. H., & Liu, A. (2008). Findings from the 2007 administration of the College Senior Survey (CSS): National aggregates. Los Angeles: Higher Education Research Institute, UCLA.

 

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.