Author: Craig Nelson

Fostering Metacognition: Right-Answer Focused versus Epistemologically Transgressive

by Craig E. Nelson at Indiana University (Contact: nelson1@indiana.edu)

I want to enrich some of the ideas posted here by Ed Nuhfer (2014 a, b, c and d) and Lauren Scharff (2014). I will start by emphasizing some key points made by Nuhfer (2014 a):

  • Instead of focusing on more powerful ways of thinking, most college instruction has thus far focused on information, concepts and discipline specific skills. I will add that even when concepts and skills are addressed they, too, are often treated as memorizable information both by students and faculty. Often little emphasis is placed on demonstrating real understanding, let alone on application and other higher-level skills.
  • “Adding metacognitive components to our assignments and lessons can provide the explicit guidance that students need. However, authoring these components will take many of us into new territory…” This is tough because such assignments require much more support for students and many of faculty members have had little or no practice in designing such support.
  • The basic framework for understanding higher-level metacognition was developed by Perry in the late 1960s and his core ideas have since been deeply validated, as well as expanded and enriched, by many other workers (e.g. Journal of Adult Development, 2004; Hoare, 2011.).
  • “Enhanced capacity to think develops over spans of several years. Small but important changes produced at the scale of single quarter or semester-long courses are normally imperceptible to students and instructors alike.”

It is helpful (e.g. Nelson, 2012, among many) to see most of college-level thinking as spanning four major levels, a truncated condensation of Perry’s 9 stages as summarized in Table 1 of Nuhfer (2014 a). Each level encompasses a different set of metacognitive skills and challenges. Individual students’ thinking is often a mix or mosaic where they approach some topics on one level and others at the next.

In this post I am going to treat only the first major level, Just tell me what I need to know (Stages 1 & 2 of Table 1 in Nuhfer, 2012 a). In this first level, students view knowledge fundamentally as Truth. Such knowledge is eternal (not just some current best model), discovered (not constructed) and objective (not temporally or socially situated). In contrast, some (but certainly not all) faculty members view what they are teaching as constructed best current model or models and as temporally and socially situated with the subjectivity that implies.

The major cognitive challenges within this first level are usefully seen as moving toward a more complete mastery of right-answer reasoning processes (Nelson, 2012), sometimes referred to as a move from concrete to formal reasoning (although the extent to which Piaget’s stages actually apply is debated). A substantial majority of entering students at most post-secondary institutions have not yet mastered formal reasoning. However, many (probably most) faculty members tacit assume that all reasonable students will quickly understand anything that is asked in terms of most right-answer reasoning. As a consequence, student achievement is often seriously compromised.

Lawson et al. (2007) showed that a simple test of formal reasoning explained about 32% of the variance in final grades in an introductory biology course and was the strongest such predictor among several options. This is quite remarkable considering that the reasoning test had no biological content and provided no measure of student effort. Although some reasoning tasks could be done by most students, an understanding of experimental designs was demonstrated largely by students who scored as having mastered formal reasoning. Similar differences in achievement have been documented for some other courses (Nelson, 2012).

Nuhfer (2014 b) and Scharff (2014) discuss studies of the associations among various measures of student thinking. From my viewpoint, their lists start too high up the thinking scale. I think that we need to start with the transitions between concrete and formal reasoning. I have provided a partial review of key aspects of this progression and of the teaching moves that have been shown to help students master more formal reasoning, as well as sources for key instruments (Nelson, 2012). I think that such mastery will turn out to be especially helpful, and perhaps essential, to more rapid development of higher level-reasoning skills.

This insight also may helps to resolve a contrast, between the experience of Scharff and her colleagues (Scharff, 2014) and Nuhfer’s perspective (2014 b). Scharff reports: “At my institution we have some evidence that such an approach does make a very measurable difference in aspects of critical thinking as measured by the CAT (Critical Thinking Assessment, a nationally normed, standardized test …).” In his responses, Nuhfer (2014 b) emphasizes that, given how we teach, there is, not surprisingly, very little change over the course an undergraduate degree in higher-order thinking. (“… the typical high school graduate is at about [Perry] level 3 2/3 and the typical college graduate is a level 4. That is only one-third of a Perry stage gain made across 4-5 years of college.”)

It is my impression that the “Critical Thinking Assessment” discussed by Scharff deals primarily with right-answer reasoning. The mastery of the skills underlying right-answer reasoning questions is largely a matter of mastering formal reasoning processes. Indeed, tests of concrete versus formal reasoning usually consist exclusively of questions that have very clear right answers. I think that several of the other thinking assessments that Nuhfer and Scharff discuss also have exclusively or primarily clear right-answers. This approach contrasts markedly with the various instruments for assessing intellectual development in the sense of Perry and related authors, none of which focuses on right-answer questions. An easily accessible instrument is given the appendices of King and Kitchener (1994).

This leads to three potentially helpful suggestions for fostering metacognition.

  • Use one of the instruments for assessing concrete versus formal reasoning as a background test for all of your metacognitive interventions. This will allow you to ask whether students who perform differently on such an assessment also perform differently on your pre- or post-assessment, or even in the course as a whole (as in Lawson et al. 2007).
  • Include interventions in your courses that are designed to help students succeed with formal, right-answer reasoning tasks. In STEM courses, teaching with a “learning cycle” approach that starts with the examination or even the generation of data is one important, generally applicable such approach.
  • Carefully distinguish between the ways that you are helping students master right-answer reasoning and the ways you are trying to foster more complex forms of reasoning. Fostering right-answer reasoning will include problem-focused reasoning, self-monitoring and generalizing right-answer reasoning processes (e.g. “Would using a matrix help me solve this problem?”).

Helping students move to deeper sophistication requires epistemologically transgressive challenges. Those who wish to pursue such approaches seriously should examine first, perhaps, Nuhfer’s (2014d) “Module 12 – Events a Learner Can Expect to Experience” and ask how one could foster each successive step.

Unfortunately, the first key step to helping students move beyond right-answer thinking requires helping them understand the ways in which back-and-white reasoning fails in one’s discipline. For this first epistemologically transgressive challenge, understanding that knowledge is irredeemably uncertain, one might want to provide enough scaffolding to allow students to make sense of readings such as: Mathematics: The Loss of Certainty (Kline, 1980); Be Forewarned: Your Knowledge is Decaying (Arbesman, 2012); Why Most Published Research Findings Are False (Ioannidis, 2005); and Lies, Damned Lies, and Medical Science (Freedman, 2010).

As an overview for students of the journey in which everything becomes a matter of better and worse ideas and divergent standards for judging better, I have had some success using a heavily scaffolded approach (detailed study guides, including exam ready essay questions, and much group work) to helping students understand Reality Isn’t What It Used to Be: Theatrical Politics, Ready-to-Wear Religion, Global Myths, Primitive Chic, and Other Wonders of the Postmodern World (Anderson,1990).

We have used various heavily scaffolded, epistemologically transgressive challenges to produce an average gain of one-third Perry stage over the course of a single semester (Ingram and Nelson, 2009). As Nuhfer (2014b) noted, this is about the gain usually produced by an entire undergraduate degree of normal instruction.

And for the bravest, most heavily motivated faculty, I would suggest In Over Our Heads: The Mental Demands of Modern Life (Kegan, 1994). Kegan attempts to make clear that each of us has our ability to think in more complex ways limited by epistemological assumptions of which we are unaware. This is definitely not a book for undergraduates nor is it one that easily embraced by most faculty members.

REFERENCES CITED

  • Hoare, Carol. Editor (2011). The Oxford Handbook of Reciprocal Adult Development and Learning. 2nd Edition. Oxford University Press.
  • Ingram, Ella L. and Craig E. Nelson (2009). Applications of Intellectual Development Theory to Science and Engineering Education. P 1-30 in Gerald F. Ollington (Ed.), Teachers and Teaching: Strategies, Innovations and Problem Solving. Nova Science Publishers.
  • Ioannidis, John (2005). “Why Most Published Research Findings Are False.” PLoS Medicine August; 2(8): e124. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ [The most downloaded article in the history of PLoS Medicine. Too technical for many first-year students even with heavy scaffolding?]
  • Journal of Adult Development (2004). [Special volume of nine papers on the Perry legacy of cognitive development.] Journal of Adult Development 11(2):59-161.
  • King, Patricia M. and Karen Strohm Kitchner (1994). Developing Reflexive Judgment: Understanding and Promoting Intellectual Growth and Critical Thinking in Adolescents and Adults. Jossey-Bass.
  • Kline, Morris (1980). Mathematics: The Loss of Certainty. Oxford University Press. [I used the summary (the Preface) in a variety of courses.]
  • Nelson, Craig E. (2012). “Why Don’t Undergraduates Really ‘Get’ Evolution? What Can Faculty Do?” Chapter 14 (p 311-347) in Karl S. Rosengren, E. Margaret Evans, Sarah K. Brem, and Gale M. Sinatra (Editors.) Evolution Challenges: Integrating Research and Practice in Teaching and Learning about Evolution. Oxford University Press. [Literature review applies broadly, not just to evolution]

The Stakes: “You’ve Been Mucking With My Mind”

by Craig Nelson Indiana University

Earlier on this blog site, Ed Nuhfer (2014, Part 1, Part 2) urged us to consider the fundamental importance of Perry’s book (1970, 1999) for understanding what we are trying to do in fostering critical thinking, metacognition and other higher order outcomes. I enthusiastically agree.

I read Perry’s book (1970) shortly after it was published. I had been teaching at IU for about five years and had seen how difficult it was to effectively foster critical thinking, even in college seniors. Perry’s book transformed my thinking and my teaching. I realized that much of my own thinking was still essentially what he might have called sophomoric. I had to decide if I was convinced enough to fundamentally change how I thought. Once I began to come to grips with that I saw that Perry’s synthesis of his students’ experiences really mattered for teaching.

Perry made clear that there were qualitatively very different ways to think. Some of these ways included what I had been trying to get my students to master as critical thinking. But Perry helped me understand more explicitly what that might mean. More importantly, perhaps, he also helped me understand how to conceptualize the limits of my approach and what kinds of critical thinking the students would need to master next if I were to be successful. At the deepest level Perry helped me see that the issues were not only how to think in sophisticated ways. The real problems are more in the barriers and costs to thinking in more sophisticated ways.

And so I began. For about five years I taught in fundamentally new ways, challenging the students’ current modes of thinking and trying to address the existential barriers to change (Nelson 1989, 1999). I then decided that perhaps I should let the students more fully in on what I was doing. I thought it might be helpful to have them actually read excepts from Perry’s book. This was a challenging thought. I was teaching a capstone course for biology majors. Perry’s “Forms of Intellectual And Ethical Development, A Scheme” was rather clearly not the usual fare for such a course. So I decided to introduce it about halfway into the course, after the students had been working within a course framework designed to foster a deep understanding of scientific thinking.

I had incorporated a full period discussion each week for which the students prepared a multiple-page worksheet analyzing a reading assignment (See the Red Pen Worksheet; Nelson 2009, 2010a). Most of the students responded very positively to Perry as a discussion assignment (Perry Discussion Assignment1; Perry Selected Passages; Ingram and Nelson 2006, 2009).

For the final discussion at the end of the course, one of the questions was approximately: “Science is always changing. I will want to introduce new readings next time. Of the ones we read, which three should I consider replacing and why, and which three should I most certainly keep and why?” Perry was the reading most frequently cited as among the “most certainly keep.” Indeed, reactions were so strongly positive that comments even included: “I personally got little from Perry but the others in my group found it so valuable that you have to keep it.”

One subsequent year I assigned Perry on a Tuesday to be read for the discussion period scheduled the next week. The next morning I arrived at my office at 8 am. One of my students was sitting outside my office on the floor. I greeted her by name and asked if she was waiting to talk to me. “Why, yes.” (She skipped “duh.” Note that we are into seriously deviant behavior here: residential campus at 8 am, no appointment, no reason to think I would be there at that time.) After a few pleasantries she announced: “I read Perry last night.” (Deviance was getting thicker: She had read the assignment immediately and a week before it was due!) “I finally understand what you are trying to do in this course, and I really like it.” (This was a fairly common reaction as Perry provided a metacognitive framework that allowed the students to more deeply understand the purpose of the assignments we had been doing2.) “And I liked Perry a lot, too.” (I am thinking: it is 8 am, she can’t be here simply to rave about an assignment.) “But, I am a bit mad at you. You have been mucking3 with my mind. College courses don’t do that! I haven’t had a course muck with my mind since high school. And, I just wanted to say that you should have warned me!” (She was a college senior.) I agreed that I should have warned her and apologized. She seemed satisfied with this and we parted on good terms.

However, I was not satisfied with this state of affairs. She had felt violated by my trying to foster changes in how she thought. And I guessed that many of my other students probably had felt at least twinges of the same feelings. This led me to ask myself: “What is the difference between indoctrination and meaningful but fair education.” I concluded that fair educational practice would require trying to make the agenda public and understood before trying to change students’ minds. This openness would require more than just assigning a reading in the middle of the semester. Thereafter, I always included a non-technical summary of Perry in my first day classes (as in Nelson 2010 b) and usually assigned Perry as the second discussion reading, having used the first discussion to start mastering the whole-period discussion process.

I would generalize my conclusion here. We instructors (almost?) always need to keep students aware of our highest-level objectives in order to avoid indoctrination rather than fair education. Fortunately, I think that this approach also will often facilitate student mastery of these objectives. It is nice when right seems to match effective, eh?

 

1I decided somewhat reluctantly to include the details of this assignment. I strongly suggest that you read this short book and select the pages and passages that seem most relevant to the students and topics that you are teaching.

2Students find it especially easy to connect with Perry’s writing. He includes numerous direct quotations from interviews with students.

3She used a different initial letter in “mucking,” as you may have expected.

————-

Ingram, Ella L. & Craig E. Nelson. 2006. Relationship between achievement and students’ acceptance of evolution or creation in an upper-level evolution course. Journal of Research in Science Teaching 43:7-24.

Ingram, Ella L. & Craig E. Nelson. 2009. Applications of intellectual development theory to science and engineering education. P 1-30 in Gerald F. Ollington (Editor), Teachers and Teaching: Strategies, Innovations and Problem Solving. Nova Science Publishers.

Nelson, Craig E. (1986). Creation, evolution, or both? A multiple model approach. P 128–159 in Robert W. Hanson (Editor), Science and Creation: Geological, Theological, and Educational Perspectives New York: MacMillan.

Nelson, Craig E. (1989). Skewered on the unicorn’s horn: The illusion of a tragic tradeoff between content and critical thinking in the teaching of science. P 17–27 in Linda Crowe (Editor), Enhancing Critical Thinking in the Sciences. Washington, DC: Society of College Science Teachers.

Nelson, Craig E. (1999). On the persistence of unicorns: The tradeoff between content and critical thinking revisited. P 168–184 in Bernice A. Pescosolido & Ronald Aminzade (Editors), The Social Worlds of Higher Education: Handbook for Teaching in a New Century. Thousand Oaks, CA: Pine Forge Press.

Nelson, Craig E. 2009. The “Red Pen” Worksheet. Quick Start Series. Center for Excellence in Learning & Teaching. Humboldt State University. 2 pp. [Edited excerpt from Nelson 2010 a.]

Nelson, Craig E. (2010 a). Want brighter, harder working students? Change pedagogies!

Examples from biology. P 119–140 in Barbara J. Millis (Editor), Cooperative Learning in Higher Education: Across the Disciplines, Across the Academy. Sterling, VA: Stylus.

Nelson, Craig E. 2010. Effective Education for Environmental Literacy. P 117-129 in Heather L. Reynolds, Eduardo S. Brondizio, and Jennifer Meta Robinson with Doug Karpa and Briana L. Gross (Editors). Teaching Environmental Literacy in Higher Education: Across Campus and Across the Curriculum. Bloomington, IN: Indiana University Press.

Nelson, Craig E. 2012. Why Don’t Undergraduates Really ‘Get’ Evolution? What Can Faculty Do? P 311-347 in Karl S. Rosengren, Sarah K. Brem, E. Margaret Evans, & Gale M. Sinatra (Editors.) Evolution Challenges: Integrating Research and Practice in Teaching and Learning about Evolution. Oxford University Press.

Nuhfer, Ed. (2104a). Metacognition for guiding students to awareness of higher-level thinking (part 1). Retrieved from https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-1/

Nuhfer, Ed. (2104b). Metacognition for guiding students to awareness of higher-level thinking (part 2). Retrieved from https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-2/

Perry, William G., Jr. (1970). Forms of Intellectual and Ethical Development in the College Years: A Scheme. New York: Holt, Rinehart, and Winston.

Perry, William G., Jr. (1999). Forms of Ethical and Intellectual Development in the College Years: A Scheme. (Reprint of the 1968 1st edition with a new introduction by Lee Knefelkamp). San Francisco: Jossey-Bass.


Everyday Metacognition

by Craig Nelson, Indiana University

This is my first post for this group. I have two goals. I want to illustrate some ways that we can use metacognition in everyday, non-academic situations. And I want to begin my posts with some reflections that are naïve. Naïve in the sense that I have not digested the work already on the blog and have not turned to the metacognitive literature. This will help me recognize later when other material really challenges my own ideas. We know from work in other contexts that making initial ideas explicit rather than tacit greatly facilitates cognitive change. In the present context, we could say that it activates metacognitive processing tools.

As always, a concrete example will make this clearer. Crouch and her associates (2004) asked “Classroom Demonstrations: Learning Tools or Entertainment?” They found that just doing a demonstration in physics had little effect on students’ understanding. It was simply entertainment. If you had a relevant misconception before the demonstration, you would probably still have it afterwards. Telling someone what is wrong with their misconception or even having them listen to a carefully constructed lecture or read a carefully constructed text is “futile,” their ideas are unlikely to change (Arons, 1976). Crouch et al. tried an alternative method. Before presenting the demonstration they asked students to write down what they thought would happen and then discuss their predictions briefly with their neighbors (thus activating prior conceptions and some relevant cognitive and metacognitive frameworks). Crouch and her associates then presented the demonstration and asked the students to compare what happened to their own predictions and to discuss the comparison with their neighbors. This led to significant conceptual change.

One of the most powerful metacognitive tools is exactly this. Ask yourself what a speaker or article or book or demonstration in class or real life is likely to say or show. Make explicit predictions. Whenever possible write them down. Then monitor the extent to which your predictions work out. Congratulate yourself when you are right and ask why you were wrong when they don’t.

A second powerful general metacognitive tool is related. Ask yourself how you do something. Then ask yourself what are some alternative ways you might do it and how you might decide which one to use in the future. For example: What pattern do you follow when you shop in the grocery store? Do you start with produce or end with produce? What else? How else might you systematically shop? And now the tough part: What criteria might favor each of the patterns? For example: Ending with heavy things such as dog-food might minimize your pushing effort but it might also risk crushing more delicate items. I try to minimize the temptation to buy junk food and processed carbohydrates generally. This means that in the grocery stores I visit, I generally shop the margins (produce, meat, diary) and avoid going through the aisles with canned food, sweets, chips and related foods unless I have something on my list that is found there. The bottom line is that for the things we do and the ways that we think, we should remember to ask, first, what are the alternatives and, second, what do we gain and lose by the ones that we choose.

I will close with two foreshadowings of points I expect to develop much more extensively later. Learning to think is a strange enterprise. Our best thinking at each point has limits that we cannot see and may not even be able to comprehend even if someone points them out to us. Misconceptions are a basic example. It is very hard to avoid taking any contradictory evidence we encounter and distorting it so that it seems to support our initial misconception (Grant, 2009). We have to make predictions or engage in strangely structured discussions (What would it take to convince you to switch to a new view if you held this misconception? Grant, 2009) or otherwise be effectively challenged. Seek out such challenges to even your most seemingly solid ideas.

This inability to see new ways of thinking applies even to the general way we perceive reality. Suppose that you think that knowledge in general and science and math in particular are based on objective truth and is likely to be eternally true. You then might have deep trouble with the titles and core ideas in for example, Kline’s Mathematics: The Loss of Certainty, Ioannidis’ Why Most Published Research Findings Are False or Freedman’s Lies, Damned Lies, and Medical Science. Even deeper challenges would be presented by, among many others, Anderson’s Reality Isn’t What It Used To Be, or Baxter Magolda’s Authoring Your Life. But each of these implicitly or explicitly presents a metacognitive framework that can be very powerful once we master it. So my final hint today for metacognitive awareness is to play Elbow’s believing game: See if you can understand how an author comes to his or her conclusions even when they seem very different from your own. Or as Russell put it, the rationale for studying the history of philosophy is to understand how an intelligent person ever came to believe such things as a tool for recognizing the limits of one’s own beliefs. We need to do this broadly, not just historically.

 References

Anderson, Walter Truett. 1990. Reality Isn’t What It Used to Be: Theatrical Politics, Ready-To-Wear Religion, Global Myths, Primitive Chic, and Other Wonders of the Postmodern World. Harpercollins.

Arons, Arnold. Arons, A. B. 1976. Cultivating the capacity for formal operations: Objectives and procedures in an introductory physical science course. American Journal of Physics 44: 834-838.

Baxter Magolda, Marcia B. 2009. Authoring Your Life: Developing an Internal Voice to Navigate Life’s Challenges. Stylus.

Crouch, Catherine H., A. P. Fagen, P. Callan and E. Mazur. 2004. “Classroom Demonstrations: Learning Tools or Entertainment?” American Journal of Physics 72:835-838.

Elbow, Peter. 1973. Writing Without Teachers. Oxford University Press.

Freedman, David. 2010. Lies, Damned Lies, and Medical Science. Atlantic. http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/?single_page=true (or http://bit.ly/11aAmt0).

Grant, B. W. 2009. Practitioner Research Improved My Students’ Understanding Of Evolution By Natural Selection In An Introductory Biology Course. Teaching Issues and Experiments in Ecology, 6(4). http://tiee.ecoed.net/vol/v6/research/grant/abstract.html

Ioannidis, John. 2005. Why Most Published Research Findings Are False. PLoS Medicine August; 2(8): e124. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ (The most downloaded article in the history of PLoS Medicine.)

Kline, Morris. 1982. Mathematics: The Loss of Certainty. Oxford University Press.

Russell, Bertrand. 1945. A History of Western Philosophy. Simon & Schuster.