So what if ‘metacognition’ is vague!

by John Draeger (SUNY Buffalo State)

When Lauren Scharff invited me to join Improve with Metacognition last year, I was only vaguely aware of what ‘metacognition’ meant. As a philosopher, I knew about various models of critical thinking and I had some inkling that metacognition was something more than critical thought, but I could not have characterized the extra bit. In her post last week, Scharff shared a working definition of ‘metacognitive instruction’ developed by a group of us involved as co-investigators on a project (Scharff, 2015). She suggested that it is the “intentional and ongoing interaction between awareness and self-regulation.” This is better than anything I had a year ago, but I want to push the dialogue further.

I’d like to take a step back to consider the conceptual nature of metacognition by applying an approach in legal philosophy used to analyze terms with conceptual vagueness. While clarity is desirable, Jeremy Waldron argues that there are limits to the level of precision that legal discourse can achieve (Waldron, 1994). This is not an invitation to be sloppy, but rather an acknowledgement that certain legal concepts are inescapably vague. According to Waldron, a concept can be vague in at least two ways. First, particular instantiations can fall along a continuum (e.g., actions can be more or less reckless, negligent, excessive, unreasonable). Second, some concepts can be understood in terms of overlapping features. Democracies, for example, can be characterized by some combination of formal laws, informal patterns of participation, shared history, common values, and collective purpose. These features are neither individually necessary nor jointly sufficient for a full characterization of the concept. Rather, a system of government counts as democratic if it has “enough” of the features. A particular democratic system may look very different from its democratic neighbor. This is in part because particular systems will instantiate the features differently and in part because particular systems might be missing some feature altogether. Moreover, democratic systems can share features with other forms of government (e.g., formal laws, common values, and collective purpose) without there being a clear boundary between democratic and non-democratic forms of government. According to Waldron, there can be vagueness within the concept of democracy itself and in the boundaries between it and related concepts.

While some might worry that the vagueness of legal concepts is a problem for legal discourse, Waldron argues that the lack of precision is desirable because it promotes dialogue. For instance, when considering whether some particular instance of forceful policing should be considered ‘excessive,’ we must consider the conditions under which force is justified and the limits of acceptability. Answering these questions will require exploring the nature of justice, civil rights, and public safety. Dialogue is valuable, in Waldron’s view, because it brings clarity to a broad constellation of legal issues even though clarity about any one of the constituents requires thinking carefully about the other elements in the constellation.

Is ‘metacognition’ vague in the ways that legal concepts can be vague? To answer this question, consider some elements in the metacognitive constellation as described by our regular Improve with Metacognition blog contributors. Self-assessment, for example, is feature of metacognition (Fleisher, 2014, Nuhfer, 2014). Note, however, that it is vague. First, self-assessments may fall along a continuum (e.g., students and instructors can be more or less accurate in their self-assessments). Second, self-assessment is composed of a variety of activities (e.g., predicting exam scores, tracking gains in performance, understanding personal weak spots and understanding one’s own level of confidence, motivation, and interest). These activities are neither individually necessary nor jointly sufficient for a full characterization of self-assessment. Rather, students or instructors are engaged in self-assessment if they engage in “enough” of these activities. Combining these two forms of vagueness, each of the overlapping features can themselves fall along a continuum (e.g., more or less accurate at tracking performance or understanding motivations). Moreover, self-assessment shares features with other related concepts such as self-testing (Taraban, Paniukov, and Kiser, 2014), mindfulness (Was, 2014), calibration (Gutierrez, 2014), and growth mindsets (Peak, 2015). All are part of the metacognitive constellation of concepts. Each of these concepts is individually vague in both senses described above and the boundaries between them are inescapably fuzzy. Turning to Scharff’s description of metacognitive instruction, all four constituent elements (i.e. ‘intentional,’ ‘ongoing interaction,’ ‘awareness,’ and ‘self-regulation’) are also vague in both senses described above. Thus, I believe that ‘metacognition’ is vague in the ways legal concepts are vague. However, if Waldron is right about the benefits of discussing and grappling with vague legal concepts (and I think he is) and if the analogy between vague concepts and the term ‘metacognition’ holds (and I think it does), then vagueness in this case should be perceived as desirable because it facilitates broad dialogue about teaching and learning.

As Improve with Metacognition celebrates its first year birthday, I want to thank all those who have contributed to the conversation so far. Despite the variety of perspectives, each contribution helps us think more carefully about what we are doing and why. The ongoing dialogue can improve our metacognitive skills and enhance our teaching and learning. As we move into our second year, I hope we can continue exploring the rich the nature of the metacognitive constellation of ideas.

References

Fleisher, Steven (2014). “Self-assessment, it’s a good thing to do.” Retrieved from https://www.improvewithmetacognition.com/self-assessment-its-a-good-thing-to-do/

Gutierrez, Antonio (2014). “Comprehension monitoring: the role of conditional knowledge.” Retrieved from https://www.improvewithmetacognition.com/comprehension-monitoring-the-role-of-conditional-knowledge/

Nuhfer, Ed (2014). “Self-Assessment and the affective quality of metacognition Part 1 of 2.”Retrieved from https://www.improvewithmetacognition.com/self-assessment-and-the-affective-quality-of-metacognition-part-1-of-2/

Peak, Charity (2015). “Linking mindset to metacognition.” Retrieved from https://www.improvewithmetacognition.com/linking-mindset-metacognition/

Scharff, Lauren (2015). “What do we mean by ‘metacognitive instruction’?” Retrieved from https://www.improvewithmetacognition.com/what-do-we-mean-by-metacognitive-instruction/

Taraban, Roman, Paniukov, Dmitrii, and Kiser, Michelle (2014). “What metacognitive skills do developmental college readers need? Retrieved from https://www.improvewithmetacognition.com/what-metacognitive-skills-do-developmental-college-readers-need/

Waldron, Jeremy (1994). “Vagueness in Law and Language: Some Philosophical Issues.” California Law Review 83(2): 509-540.

Was, Chris (2014). “Mindfulness perspective on metacognition. ”Retrieved from https://www.improvewithmetacognition.com/a-mindfulness-perspective-on-metacognition/

What Do We Mean by “Metacognitive Instruction”?

by Lauren Scharff (U.S. Air Force Academy*)

Many of you are probably aware of the collaborative, multi-institutional metacognitive instruction research project that we initiated through the Improve with Metacognition site. This project has been invigorating for me on many levels. First, through the process of developing the proposal, I was mentally energized. Several of us had long, thoughtful conversations about what we meant when we used the term “metacognitive instruction” and how these ideas about instruction “mapped” to the concept of “metacognitive learning.” These discussions were extensions of some early blog post explorations, What do we mean when we say “Improve with metacognition”? (Part 1 and Part 2). Second, my involvement in the project led me to (once again) examine my own instruction. Part of this self-examination happened as a natural consequence of the discussions, but also it’s happening in an ongoing manner as I participate in the study as an intervention participant. Good stuff!

For this post, I’d like to share a bit more about our wrangling with what we meant by metacognitive instruction as we developed the project, and I invite you to respond and share your thoughts too.

Through our discussions, we ultimately settled on the following description of metacognitive instruction:

Metacognitive instructors are aware of what they are doing and why. Before each lesson, they have explicitly considered student learning goals and multiple strategies for achieving those goals. During the lesson, they actively monitor the effectiveness of those strategies and student progress towards learning goals. Through this pre-lesson strategizing and during lesson monitoring awareness, a key component of metacognition, is developed; however, awareness is not sufficient for metacognition. Metacognitive instructors also engage in self-regulation. They have the ability to make “in-the-moment”, intentional changes to their instruction during the lesson based on a situational awareness of student engagement and achievement of the learning objectives — this creates a responsive and customized learning experience for the student.

One of the questions we pondered (and we’d love to hear your thoughts on this point), is how these different constructs were related and / or were distinct. We came to the conclusion that there is a difference between reflective teaching, self-regulated teaching, and metacognitive instruction/teaching.

More specifically, a person can reflect and become aware of their actions and their consequences, but at the same time not self-regulate to modify behaviors and change consequences, especially in the moment. A person can also self-regulate / try a new approach / be intentional in one’s choice of actions, but not be tuned in / aware of how it’s going at the moment with respect to the success of the effort. (For example, an instructor might commit to a new pedagogical approach because she learned about it from a colleague. She can implement that new approach despite some personal discomfort due to changing pedagogical strategies, but without conscious and intentional awareness of how well it fits her lesson objectives or how well it’s working in the moment to facilitate her students’ learning.) Metacognition combines the awareness and self-regulation pieces and increases the likelihood of successfully accomplishing the process (teaching, learning, or other process).

Thus, compared to other writings we’ve seen, we are more explicitly proposing that metacognition is the intentional and ongoing interaction between awareness and self-regulation. Others have generally made this claim about metacognitive learning without using the terms as explicitly. For example, “Simply possessing knowledge about one’s cognitive strengths or weaknesses and the nature of the task without actively utilizing this information to oversee learning is not metacognitive.” (Livingston, 1997). But, in other articles on metacognition and on self-regulated learning, it seems like perhaps the metacognitive part is the “thinking or awareness” part and the self-regulation is separate.

What do you think?

——————
Livingston, J. A. (1997). Metacognition: An Overview. Unpublished manuscript, State University of New York at Buffalo. http://gse.buffalo.edu/fas/shuell/cep564/metacog.htm

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.

Executive Function: Can Metacognitive Awareness Training Improve Performance?

by Antonio Gutierrez, Georgia Southern University

In a recent meta-analysis of 67 research studies that utilize an intervention targeted at enhancing metacognitive awareness, Jacob and Parkinson (in press) argue that metacognitive interventions aimed at improving executive function processes are not as effective at improving student achievement as once believed by scholars and practitioners alike. In essence, the evidence in support of robust effects of these types of interventions in improving achievement is inconclusive. While descriptive research studies continue to report high associations between metacognitive awareness and performance or achievement measures, Jacob and Parkinson argue that the experimental evidence supporting a strong role of metacognitive training in improving student performance is scant. I have recently pondered a similar dilemma with research on the effect of metacognitive monitoring training on students’ performance, confidence judgments but especially calibration. The literature on these topics converges on the finding that metacognitive monitoring training improves performance and confidence in performance judgments but not necessarily calibration (see e.g., Bol et al., 2005; Gutierrez & Schraw, 2015; Hacker et al., 2008).

While Jacob and Parkinson’s meta-analysis is illuminating, I wonder whether, like the calibration literature, the conclusion that executive function interventions are not as effective at improving achievement may be due to very different conceptualizations of the constructs under investigation. In the case of calibration, the mixed findings may be due to the fact that the metacognitive monitoring interventions were not likely targeting the same thing. For instance, some interventions may have been targeting a reduction in calibration errors (overconfidence and underconfidence), others may have been targeting improvement in calibration accuracy, whereas yet others may have been targeting both, whether intentionally or unintentionally. Because these interventions were targeting different aspects of calibration, it could be that the inconclusive findings were due to a confounding of these various dimensions of calibration … comparing apples to oranges, if you will. Could the lack of robust effects of executive function interventions on achievement be due to a similar phenomenon? What if these studies were not targeting the same executive function processes, in which case they would not be as directly comparable as at first glance? Jacob and Parkinson’s (in press) study may lead some to believe that there is little to be gained in investing time and effort in executive function interventions. However, before we abandon these interventions, perhaps we should consider developing executive function interventions that are more specific and finer grained such as by targeting very specific aspects of the executive function rather than a more general approach.

References
Bol, L., Hacker, D. J., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy, and performance. The Journal of Experimental Education, 73, 269-290.

Gutierrez, A. P., & Schraw, G. (2015). Effects of strategy training and incentives on students’ performance, confidence, and calibration. The Journal of Experimental Education: Learning, Instruction, and Cognition. Advance online publication. doi: 10.1080/00220973.2014.907230

Hacker, D. J., Bol, L., & Bahbahani, K. (2008). Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style. Metacognition Learning, 3, 101-121.

Jacob, R., & Parkinson, J. (in press). The potential for school-based interventions that target executive function to improve academic achievement: A review. Review of Educational Research. Advance online publication. doi: 10.3102/0034654314561338

The Metacognitive Syllabus!

By Aaron S. Richmond, Ph.D.
Metropolitan State University of Denver

This blog may be like no other in Improve with Metacognition (IwM). I am asking you, the readers to actively participate. Yes, I mean YOU, YOU, and YOU☺. But let me clarify—I do not ask rhetorical questions. As such, please respond using the comment function in IwM or Tweet your answer to the three questions in this blog.

Question #1: How can we use the syllabus as a metacognitive tool?
As delineated by scores of researchers and teachers, the syllabus can be many things. The syllabus can be a contract (Slattery & Carlson, 2005). These elements of the syllabus typically include policies on attendance, late work, ethics, grading, etc. The syllabus can also be a permanent record (Parkes & Harris, 2002). Permanent record elements of a syllabus include course objectives, assessment procedures, course description, and course content. The syllabus is also a communication device that can set the tone for your class and is an opportunity to gain your students trust and respect by modeling your pedagogical beliefs (Bain, 2004) .

As the syllabus can be many things, the syllabus, it is very possible that the syllabus can serve as a metacognitive tool. Several researchers suggest that the syllabus is also a cognitive map (Parkes & Harris, 2002) and a learning tool (Matejka & Kurke, 1994). These elements typically include a description of how to succeed in the course, common pitfalls and misconceptions that occur in the course, campus resources that can assist the students in learning (e.g., writing center), a teaching philosophy, and embedded explanations of class assignments, structure, and student learning. If we consider the syllabus in this context, I believe that we can easily incorporate metacognitive elements. For instance, in my personal teaching philosophy, I specifically mention my focus on improving metacognition. Another example is that I have at least one student learning objective that is megacogntively based with assignments designed to assess this objective. For example, Students will understand what metacognition is and how it improves their own learning (assessed by experiential learning assignment 1 and comprehensive exam). Or Students will understand what it means to develop a culture of metacognition in the classroom (assessed by classroom observation and mid-term exam). Finally, I actively incorporate course content on learning strategies and the metacognitive explanations for those strategies which sets the tone for the importance of metacognition in the class.

Question #2: How are you using the syllabus as a metacognitive tool?
I really want to hear from you on how you may be using the syllabus as a metacognitive tool. For example, what specific statements do you include related to metacognition goals? What assignments do you mention that link to metacognitive development?

Question #3: If the syllabus can be used as a metacognitive tool, how do we know it is effective?
What is your answer to this question? My answer centers on the Scholarship of Teaching and Learning. That is, we don’t have empirical evidence yet to say that the syllabus is a metacognitive tool. That doesn’t mean that it can’t be or isn’t already in practice. But I think you(we) should take up this challenge and investigate this issue. The syllabus can have profound impact on how student learning, instruction, and student ratings of instruction (Richmond, Becknell, Slattery, Morgan, & Mitchell, 2015; Saville, Zinn, Brown, & Marchuk, 2010). so let’s investigate how to improve the syllabus through metacognition.

UsCourse syllabi can be a metacognitive tool. Share on X

References
Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press.
Matejka, K., & Kurke, L. B. (1994). Designing a great syllabus. College Teaching, 42(3), 115-117. doi:10.1080/87567555.1994.9926838
Parkes, J., & Harris, M. B. (2002). The purposes of a syllabus. College Teaching, 50(2), 55-61. doi:10.1080/87567550209595875
Richmond, A. S., Becknell, J., Slattery, J., Morgan, R., & Mitchell, N. (2015, August). Students’ perceptions of a student-centered syllabus: An experimental analysis. Poster presented the annual meeting of the American Psychological Association, Toronto, Canada.
Saville, B. K., Zinn, T. E., Brown, A. R., & Marchuk, K. A. (2010). Syllabus detail and students’ perceptions of teacher effectiveness. Teaching of Psychology, 37, 186-189. doi:10.1080/00986283.2010.488523
Slattery, J. M., & Carlson, J. F. (2005). Preparing an effective syllabus: Current best practices. College Teaching, 53, 159-164. doi:10.3200/CTCH.53.4.159-164

Exploring the Developmental Progression of Metacognition

by Sarah L. Bunnell at Ohio Wesleyan University (slbunnel@owu.edu)

As a developmental psychologist, it is difficult to consider student learning (and my own learning as well) without a strong nod to developmental process. Metacognition, as has been described by many others on this blog and in other venues (e.g., Baxter-Magolda, 1992; Flavell, 1979; Kuhn, 1999; Perry, 1970), requires the cognitive skills of reflection, connection, evaluation, and revision. Metacognitive acts are initially quite cognitively demanding, and like most conscious cognitive processes, require practice to become more automatic or at least less consuming of cognitive resources. In addition to examining how students acquire the tools required for the hard work of metacognition, I have also been interested in whether there are developmental differences in students’ ability to make connections and reflections across the college years.

I recently conducted two examinations of metacognitive development; the first project involved my Introductory Psychology course, which enrolls primarily first year students, and the second project involved my Adolescent Psychology course, which enrolls primarily sophomore-level students. Below, I will provide a brief summary of each study and then discuss what I see as some take-home points and next-steps for inquiry.

In the Introductory Psychology course (n = 45), each student completed a metacognitive portfolio (hosted through the MERLOT website; http://eportfolio.merlot.org/) throughout the semester. In this portfolio, students responded to a series of prompts to reflect on their current thinking about course concepts and the ways in which these concepts play out in their own lives. At the end of the semester, students were asked to review their responses, identify any responses that they would now change, and explain why they would now alter their responses. They were also asked to describe how they thought their thinking had changed over the course of the semester.

Given the large body of work on the learning benefits associated with metacognition, I was not surprised that students who wanted to change a greater number of their responses performed significantly better on the final exam than did students who identified fewer points of change. More surprising, however, was the finding that students who did well on the final exam were significantly more likely to have endorsed changes in their thinking about themselves as opposed to changes in their thinking about others. A year after this class ended, I contacted these same students again, and I asked them to reflect on their thinking at the end of the course relative to their thinking about Psychology and themselves now. Of note, an analysis of these responses indicated that the students who were high performers on the final exam and in the course overall were no longer reporting many self-related metacognitive links. Instead, these students were significantly more likely to say that they now had a greater understanding of others than they did before. Thus, there was a powerful shift over time in the focus of metacognition from self to other.

In my Adolescent Psychology course (n = 35), students conduct a semi-structured interview of an adolescent, transcribe the interview, and then analyze the interview according to developmental theory. This assignment is designed to foster connection and application, and I have compelling evidence indicating that this experience enhances learning. What was less clear to me, however, is whether participating in this course and in the interview paper activity contributes to students’ metacognitive awareness of self? To address this question, I implemented a pre-post test design. On the first day of class, students were asked, “Are you currently an adolescent? Please explain your answer.” To answer this question, one must consider multiple ways in which we may conceptualize adolescence (i.e., age, legal responsibility, physical maturity, financial responsibility); as you can clearly see, the lens we apply to ourselves and others leads to quite varied views of when adolescence ends and adulthood begins! At the end of the term, students were again asked the same question, plus an additional prompt that asked them to reflect on how their thinking about themselves had changed across the semester.

On Day 1, 17 students endorsed currently being an adolescent, 16 students reported no longer being an adolescent, and 2 students said they did not feel that they had enough information to respond. It is important to note that all students in the course were between the ages of 18 and 21 years and as such, all were technically late adolescents. On the last day of class, 21 class members labeled themselves as adolescents, 4 students said that they did not consider themselves to be adolescents, and 5 said that they were an adolescent in some contexts of their life and not others. As an example of a contextual way of thinking, one student said: “I believe that neurologically I am still an adolescent because I am below the age of 25 and therefore do not have a fully developed frontal lobe, which can alter decision making, and from a Piagetian standpoint I believe I am out of adolescence because I have reached the formal operational stage of development and possibly even beyond that. Overall though, I believe that I can’t fully define myself as an adolescent or not because there are so many factors in play.”

I examined these group-level differences in terms of course performance from a number of angles, and two interesting patterns emerged. First, students who adopted a more context-dependent view of self did significantly better on the application-based, cumulative final exam than did students who held an absolute view of self. This first finding is consistent with the work on Marcia Baxter-Magolda (1992), William Perry (1970), and others, which views contextual knowing as a complicated and mature form of meta-knowing. Second, students who changed their view of themselves across the semester conducted significantly more advanced analyses of the interview content relative to those whose view of self did not change. Thus, the students who displayed greater advances in metacognition were better able to apply these reflections and connections to themselves and, in turn, to the lives of others.

Taken together, this work suggests to me that the ability to engage in metacognitive reflection and connection may initially develop in a self-focused way and then, following additional experience and metacognitive skill attainment, extend beyond the self. Please note that I am careful to suggest that the ability of other-related connection emerges following experience and the acquisition of lower-level preparatory skills, rather than merely the passage of time, even though there is clearly a temporal dimension at play. Instead, as Donald Baer warned us, age is at best a proxy for development; at the most extreme, development is “age-irrelevant” (Baer, 1970). Why do students demonstrate improved metacognition across the college years? It is certainly not merely because the days have ticked by. Instead, these advances in thinking, as well as students’ willingness to refine their thinking about the self, are supported and constructed by a range of experiences and challenges that their college experience affords. To understand age- or college-level changes in thinking, therefore, we should focus on the developmental tasks and experiences that support this development. I hope that my lines of inquiry contribute in small part to this process.

References

Baer, D. M. (1970). An age-irrelevant concept of development. Merrill-Palmer Quarterly, 16, 238-245.

Baxter Magolda, M. B. (1992). Students’ epistemologies and academic experiences: implications for pedagogy, Review of Higher Education, 15, 265-87.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906 – 911.

Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28, 16-25.

Perry, William G., Jr. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Rinehart, & Winston.