Metacognition and Mindset for Growth and Success: Part 1 – Understanding the Metacognitive Connections between Self-Assessment and Mindset

by Steven Fleisher, California State University
Michael Roberts, DePauw University
Michelle Mason, University of Wyoming
Lauren Scharff, U. S. Air Force Academy
Ed Nuhfer, Guest Editor, California State University (retired)

When I first entered graduate school, I was flourishing. I was a flower in full bloom. My roots were strong with confidence, the supportive light from my advisor gave me motivation, and my funding situation made me finally understand the meaning of “make it rain.” But somewhere along the way, my advisor’s support became only criticism; where there was once warmth, there was now a chill, and the only light I received came from bolts of vindictive denigration. I felt myself slowly beginning to wilt. So, finally, when he told me I did not have what it takes to thrive in academia, that I wasn’t cut out for graduate school, I believed him… and I withered away. (actual co-author experience)

schematic of person with band aid and flowers growing who is facing other people — Image by Moondance from Pixabay

After reading the entirety of this two-part blog entry, return and read the shared experience above once more. You should find that you have an increased ability to see the connections there between seven elements: (1) affect, (2) cognitive development, (3) metacognition, (4) self-assessment, (5) feedback, (6) privilege, and (7) mindset.

The study of self-assessment as a valid component of learning, educating, and understanding opens up fascinating areas of scholarship for new exploration. This entry draws on the same paired-measures research described in the previous blog entries of this series. Here we explain how measuring self-assessment informs understanding of mindset and feedback. Few studies connect self-assessment with mindset, and almost none rest on a sizeable validated data set.

Mindset, self-assessment, and privilege

Mindset theory proposes that individuals lean toward one of two mindsets (Dweck, 2006) that differ based on internalized beliefs about intelligence, learning, and academics. According to Dweck and others, people fall along a continuum that ranges from having a fixed mindset defined by a core belief that their intelligence and thinking abilities remain fixed, and effort cannot change them. In contrast, having a growth mindset comes with the belief that, through their effort, people can expand and improve their abilities to think and perform (Figure 1).

Indeed, a growth mindset has support in the stages of intellectual, ethical, and affective development discovered by Bloom & Krathwohl and William Perry mentioned earlier in this series. However, mindset theory has evolved into making broader claims and advocating that being in a state of growth mindset also enhances performance in high-stakes functions such as leadership, teaching, and athletics.

Figure 1. Fixed – growth mindset tendencies. (From https://trainugly.com/portfolio/growth-mindset/)

Do people choose their mindset or do their experiences place them in their positions on the mindset continuum? Our Introduction to this series disclosed that people’s experiences from degrees of privilege influence their positioning along the self-assessment accuracy continuum, and self-assessment has some commonalities with mindset. However, a focused, evidence-based study of privilege on determining mindset inclination seems lacking.

Our Introduction to this series indicated that people do not choose their positions along the self-assessment continuum. People’s cumulative experiences place them there. Their positions result from their individual developmental histories, where degrees of privilege influence the placement through how many experiences an individual has that are relevant and helpful to building self-assessment accuracy. The same seems likely for determining positions along the mindset continuum.

Acting to improve equity in educational success

Because the development during pre-college years primarily occurs spontaneously by chance rather than by design, people are rarely conscious of how everyday experiences form their dispositions. College students are unlikely even to know their positions on either continuum unless they receive a diagnostic measure of their self-assessment accuracy or their tendency toward a growth or a fixed mindset. Few get either diagnosis anywhere during their education.

Adapting a more robust growth mindset and acquiring better self-assessment accuracy first requires recognizing that these dispositions exist. After that, devoting systematic effort to consciously enlisting metacognition during learning disciplinary content seems essential. Changing the dispositions takes longer than just learning some factual content. However, the time required to see measurable progress can be significantly reduced by a mentor/coach who directs metacognitive reflection and provides feedback.

Teaching self-assessment to lower-division undergraduates by providing numerous relevant experiences and prompt feedback is a way to alleviate some of the inequity produced by differential privilege in pre-college years. The reason to do this early is to allow students time in upper-level courses to ultimately achieve healthy self-efficacy and graduate with the capacity for lifelong learning. A similar reason exists for teaching students the value of affect and growth mindset by providing awareness, coaching, and feedback. Dweck describes how achieving a growth mindset can mitigate the adverse effects of inequity in privilege.

Recognizing good feedback

Dweck places high value on feedback for achieving the growth mindset. The Figure 1 in our guest series’ Introduction also emphasizes the importance of feedback in developing self-assessment accuracy and self-efficacy during college.

Depending on a person’s beliefs about their particular skill to address a challenge, they will respond in predictable ways when a skill requires effort, when it seems challenging, when effort affects performance, and when feedback informs performance. Those with a fixed mindset realize that feedback will indicate imperfections, which they take as indicative of their fixed ability rather than as applicable to growing their ability. To them, feedback shames them for their imperfections, and it hurts. They see learning environments as places where stressful competitions occur between their own and others’ fixed abilities. Affirmations of success rest in grades rather than growing intellectual ability.

Those with a growth mindset value feedback as illuminating the opportunities for advancing quickly in mastery during learning. Sharing feedback with peers in their learning community is a way to gain pleasurable support from a network that encourages additional effort. There is little doubt which mindset promotes the most enjoyment, happiness, and lasting friendships and generates the least stress during the extended learning process of higher education.

Dweck further stresses the importance of distinguishing feedback that is helpful from feedback that is damaging. Our lead paragraph above revealed a devastating experience that would influence any person to fear feedback and seek to avoid it. A formative influence that disposes us to accept or reject feedback likely lies in the nature of feedback that we received in the past. A tour through traits of Dweck’s mindsets suggests many areas where self-perceptions can form through just a single meaningful feedback event.

Australia’s John Hattie has devoted his career to improving education, and feedback is his specialty area. Hattie concluded that feedback is “…the most powerful single moderator that enhances achievement” and noted in this University of Auckland newsletter “…arguably the most critical and powerful aspect of teaching and learning.”

Hattie and Timperley (2007) synthesized many years of studies to determine what constitutes feedback helpful to achievement. In summary, valuable feedback focuses on the work process, but feedback that is not useful focuses on the student as a person or their abilities and communicates evaluative statements about the learner rather than the work. Hattie and Dweck independently arrived at the same surprising conclusion: even praise directed at the person, rather than focusing on the effort and process that led to the specific performance, reinforces a fixed mindset and is detrimental to achievement.

Professors seldom receive mentoring on how to provide feedback that would promote growth mindsets. Likewise, few students receive mentoring on how to use peer feedback in constructive ways to enhance one another’s learning.

Takeaways

Scholars visualize both mindset and self-assessment as linear continuums with two respective dispositions at each of the ends: growth and fixed mindsets and perfectly accurate and wildly inaccurate self-assessments. In this Part 1, we suggest that self-assessment and mindset have surprisingly close connections that scholars have scarcely explored.

Increasing metacognitive awareness seems key to tapping the benefits of skillful self-assessment, mindset, and feedback and allowing effective use of the opportunities they offer. Feedback seems critical in developing self-assessment accuracy and learning through the benefits of a growth mindset. We further suggest that gaining benefit from feedback is a learnable skill that can influence the success of individuals and communities. (See Using Metacognition to Scaffold the Development of a Growth Mindset, Nov 2022.)

In Part 2, we share findings from our paired measures data that partially explain the inconsistent results that researchers have obtained between mindset and learning achievement. Our work supports the validity of mindset and its relationship to cognitive competence. It allows us to make recommendations for faculty and students to apply this understanding to their advantage.

References

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Heft, I. & Scharff, L. (July 2017). Aligning best practices to develop targeted critical thinking skills and habits. Journal of the Scholarship of Teaching and Learning, Vol 17(3), pp. 48-67. http://josotl.indiana.edu/article/view/22600

Isaacson, Randy M., and Frank Fujita. 2006. “Metacognitive Knowledge Monitoring and Self-Regulated Learning: Academic Success and Reflections on Learning.” Journal of Scholarship of Teaching and learning6, no. 1: 39–55. Retrieved from https://eric.ed.gov/?id=EJ854910

Yeager, D. S., & Dweck, C. S. (2020). What can be learned from growth mindset controversies? American Psychologist, 75(9), 1269–1284. https://doi.org/10.1037/amp0000794

Metacognitive Self-assessment in Privilege and Equity – Part 2: Majority Privilege in Scientific Thinking

Ed Nuhfer, California State University (Retired)
Rachel Watson, University of Wyoming
Cinzia Cervato, Iowa State University
Ami Wangeline, Laramie County Community College

Being in the majority carries the privilege of empowerment to set the norms for acceptable beliefs. Minority status for any group invites marginalization by the majority simply because the group appears different from the familiar majority. Here, we explore why this survival mechanism (bias) also operates when a majority perceives an idea as different and potentially threatening established norms.

Young adult learners achieve comfort in ways of thinking and explaining the world from their experiences obtained during acculturation. Our Introduction stressed how these experiences differ in the majority and minority cultures and produce measurable effects. Education disrupts established states of comfort by introducing ideas that force reexaminations that contradict earlier beliefs established from experiences.

Even the kind of college training that promotes only growing cognitive expertise is disruptive but more critical; research verifies that the disruptions are felt. While discovering the stages of intellectual development, William Perry Jr. found that, for some learners, the feelings experienced during transitions toward certain higher stages of thinking were so discomforting that the students ceased trying to learn and withdrew. Currently, about a third of first-year college students drop out before their sophomore year.

Educating for self-assessment accuracy to gain control over bias

We believe that the same survival mechanisms that promote prejudice and suppress empathizing and understanding different demographic groups also cripple understanding in encounters with unfamiliar or contrarian ideas. In moments that introduce ideas disruptive to beliefs or norms, unfamiliar ideas become analogous to unfamiliar groups—easily marginalized and thoughtlessly devalued in snap judgments. Practice in doing self-assessment when new learning surprises us should be valuable for gaining control over the mechanism that triggers our own polarizing bias.

Earlier (Part 2 entry on bias), we recommended teaching students to frequently self-assess, “What am I feeling that I want to be true, and why do I have that feeling?” That assignment ensures that students encounter disruptive surprises mindfully by becoming aware of affective feelings involved in triggering their bias. Awareness gives the greater control over self needed to prevent being captured by a reflex to reject unfamiliar ideas out of hand or to marginalize those who are different.

Teaching by employing self-assessment routinely for educating provides the prolonged relevant practice with feedback required for understanding self. Educating for self-assessment accuracy constitutes a change from training students to “know stuff” to educating students to know how they can think to understand both “stuff” and self.

When the first encounter with something or someone produces apprehension, those who gain a capacity for self-assessment accuracy from practice can exercise more control over their learning through recognizing the feeling that accompanies incipient activation of bias in reaction to discomfort. Such self-awareness allows a pause for reflecting on whether enlisting this vestigial survival mechanism serves understanding and can prevent bias from terminating our learning and inducing us to speak or act in ways that do not serve to understand.

Affect, metacognition, and self-assessment: minority views of contrarian scholars

We address three areas of scholarship relevant to this guest-edited series to show how brain survival mechanisms act to marginalize ideas that contradict an established majority consensus.

Our first example area involves the marginalization of the importance of affect by the majority of behavioral scientists. Antonio Damasio (1999, p. 39) briefly described this collective marginalization:

“There would have been good reason to expect that, as the new century started, the expanding brain sciences would make emotion part of their agenda…. But that…never came to pass. …Twentieth Century science…moved emotion back into the brain, but relegated it to the lower neural strata associated with ancestors whom no one worshipped. In the end, not only was emotion not rational, even studying it was probably not rational.“

A past entry in Improve with Metacognition (IwM) also noted the chilling prejudice against valuing affect during the 20th Century. Benjamin Bloom’s Taxonomy of the Affective Domain (Krathwohl et al. 1964) received an underwhelming reception from educators who had given unprecedented accolades to the team’s earlier volume on Taxonomy of the Cognitive Domain (Bloom, 1956). Also noted in that entry was William G. Perry’s purposeful avoidance of referring to affect in his landmark book on intellectual and ethical development (Perry, 1999). The Taxonomy of the Affective Domain also describes a developmental model that maps onto the Perry model of development much better than Bloom’s Taxonomy of the Cognitive Domain.

Our second example involved resistance against valuing metacognition. Dunlosky and Metcalfe (2009) traced this resistence to French philosopher Auguste Comte (1798-1854), who held that an observer trying to observe self was engaged in an impossible task like an eye trying to see itself by looking inwardly. In the 20th Century, the behaviorist school of psychology gave new life to Comte’s views by professing that individuals’ ability to do metacognition, if such an ability existed, held little value. According to Dunlosky and Metcalfe (2009, p. 20), the behaviorists held “…a stranglehold on psychology for nearly 40 years….” until the mid-1970s, when the work of John Flavell (see Flavell, 1979) made the term and concept of metacognition acceptable in academic circles.

Our third example area involves people’s ability to self-assess. “The Dunning-Kruger effect” holds that most people habitually overestimate their competence, with those least competent holding the most overly inflated views of their abilities and those with real expertise revealing more humility by consistently underestimating their abilities by modest amounts. Belief in “the effect” permeated many disciplines and became popular among the general public. As of this writing, a Google search brought up 1.5 million hits for the “Dunning Kruger effect.” It still constitutes the majority view of American behavioral scientists about human self-assessment, even after recent work revealed that the original mathematical arguments for “the effect” were untenable.

Living a scholars’ minority experience

Considering prejudice against people and bias against new ideas as manifestations of a common, innate survival mechanism obviates fragmentation of these into separate problems addressed through unrelated educational approaches. Perceiving that all biases are related makes evident that the tendency to marginalize a new idea will certainly marginalize the proponents of an idea.

Seeing all bias as related through a common mechanism supports using metacognition, particularly self-assessment, for gaining personal awareness and control over the thoughts and feelings produced as the survival mechanism starts to trigger them. Thus, every learning experience providing discomfort in every subject offers an opportunity for self-assessment practice to gain conscious control over the instinct to react with bias.

Some of the current blog series authors experienced firsthand the need for higher education professionals to acquire such control. When publishing early primary research in the early 1990s, we were naively unaware of majority consensus, had not yet considered bias as a survival reaction, and we had not anticipated marginalization. Suggesting frequent self-assessments as worthwhile teaching practices in the peer-reviewed literature brought reactions that jolted us from complacency into a new awareness.

Scholars around the nation, several of them other authors of this blog series, read the guest editor’s early work, introduced self-assessment in classes and launched self-assessment research of their own. Soon after, many of us discovered disparagements at the departmental, college, and university levels, and even at professional meetings followed for doing so. Some disparagements led to damaged careers and work environments.

The bias imparted by marginalization led to our doubting ourselves. Our feelings for a time were like those of the non-binary gender group presented in the earlier Figure 1 in the previous Part 1 on privilege: We “knew our stuff,” but our feelings of competence in our knowledge lagged. Thanks to the feedback from the journal peer-reviewers of Numeracy, we now live with less doubt in ourselves. For those of us who weathered the storm, we emerged with greater empathy for minority status and minority feelings and greater valuing of self-assessment.

Self-assessment, a type of metacognition employing affect, seems in a paradigm change that recapitulates the history of affect and metacognition. Our Numeracy articles have achieved over 10,000 downloads, and psychologists in Europe, Asia, and Australia now openly question “the effect” (Magnus and Peresetsky, 2021; Kramer et al., 2022; Hofer et al., 2022; Gignac, 2022) in psychology journals. The Office of Science and Society at McGill University in Canada reached out to the lay public (Jarry, 2020) to warn how new findings require reevaluating “the effect.” We recently discovered that paired measures could even unearth unanticipated stress indicators among students (view section at time 21.38 to 24.58) during the turbulent times of COVID and civil disruption.

Takeaways

Accepting teaching self-assessment as good practice for educating and self-assessment measures as valid assessments open avenues for research that are indeed rational to study. After one perceives bias as having a common source, developing self-assessment accuracy seems a way to gain control over personal bias that triggers hostility against people and ideas that are not threatening, just different.

“Accept the person you are speaking with as someone who has done amazing things” is an outstanding practice stressed at the University of Wyoming’s LAMP program. Consciously setting one’s cognition and affect to that practice erases all opportunities for marking anyone or their ideas for inferiority.

References

Bloom, B.S. (Ed.). (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. New York, NY: Longman.

Damasio, A. (1999). The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist 34, 906-911.

Gignac, Gilles E. (2022). The association between objective and subjective financial literacy: Failure to observe the Dunning-Kruger effect. Personality and Individual Differences 184: 111224. https://doi.org/10.1016/j.paid.2021.111224

Hofer, G., Mraulak, V., Grinschgl, S., & Neubauer, A.C. (2022). Less-Intelligent and Unaware? Accuracy and Dunning–Kruger Effects for Self-Estimates of Different Aspects of Intelligence. Journal of Intelligence, 10(1). https://doi.org/10.3390/jintelligence10010010

Kramer, R. S. S., Gous, G., Mireku, M. O., & Ward, R. (2022). Metacognition during unfamiliar face matching. British Journal of Psychology, 00, 1– 22. https://doi.org/10.1111/bjop.12553

Krathwohl, D.R., Bloom, B.S. and Masia, B.B. (1964) Taxonomy of Educational Objectives: The Affective Domain. New York: McKay.

Magnus, Jan R., and Peresetsky, A. (October 04, 2021). A statistical explanation of the Dunning-Kruger effect. Tinbergen Institute Discussion Paper 2021-092/III, http://dx.doi.org/10.2139/ssrn.3951845

Nicholas-Moon, Kali. (2018). “Examining Science Literacy Levels and Self-Assessment Ability of University of Wyoming Students in Surveyed Science Courses Using the Science Literacy Concept Inventory with Expanded Inclusive Demographics.” Master’s thesis, University of Wyoming.

Perry, W. G. Jr. (1999). Forms of Ethical and Intellectual Development in the College Years. San Francisco, CA: Jossey-Bass (a reprint of the original 1968 work with minor updating).

Tarricone, P. (2011). The Taxonomy of Metacognition (1st ed.). Psychology Press. 288p. https://doi.org/10.4324/9780203830529

Metacognitive Self-assessment in Privilege and Equity – Part 1 Conceptualizing Privilege and its Consequences

by Rachel Watson, University of Wyoming
Ed Nuhfer, California State University (Retired)
Cinzia Cervato, Iowa State University
Ami Wangeline, Laramie County Community College

Demographics of metacognition and privilege

The Introduction to this series asserted that lives of privilege in the K-12 years confer relevant experiences advantageous to acquire the competence required for lifelong learning and entry into professions that require college degrees. Healthy self-efficacy is necessary to succeed in college. Such self-efficacy comes only after acquiring self-assessment accuracy through practice in using the relevant experiences for attuning the feelings of competence with demonstrable competence. We concur with Tarricone (2011) in her recognition of affect as an essential component of the self-assessment (or awareness) component of metacognition: the “‘feeling of knowing’ that accompanies problem-solving, the ability to distinguish ideas about which we are confident….”

A surprising finding from our paired measures is how closely the mean self-assessments of performance of groups of people track with their actual mean performances. According to the prevailing consensus of psychologists, mean self-assessments of knowledge are supposed to confirm that people, on average, overestimate their demonstrable knowledge. According to a few educators, self-reported knowledge is supposed to be just random noise with no meaningful relationship to demonstrable knowledge. Data published in 2016 and 2017 in Numeracy from two reliable, well-aligned instruments revealed that such is not the case. Our reports in Numeracy shared earlier on this blog (see Figures 2 and 3 at this link) confirm that people, on average, self-assess reasonably well.

In 2019, by employing the paired measures, we found that particular groups of peoples’ average competence varied measurably, and their average self-assessed competence closely tracked their demonstrable competence. In brief, different demographic groups, on average, not only performed differently but also felt differently about their performance, and their feelings were accurate.

Conceptualizing privilege and its consequences

Multiple systems (structural, attitudinal, institutional, economic, racial, cultural, etc.) produce privilege, and all individuals and groups experience privilege and disadvantage in some aspects of their lives. We visualize each system as a hierarchical continuum, along which at one end lie those systematically marginalized/minoritized, and those afforded the most advantages lie at the other. Because people live and work within multiple systems, each person likely operates at different positions along different continuums.

Those favored by privilege are often unaware of their part in maintaining a hierarchy that exerts its toll on those of lesser privilege. As part of our studies of the effects on those with different statuses of privilege, we discovered that instruments that can measure cognitive competence and self-assessments of their competence offer richer assessments than competency scores. They also inform us about how students feel and how accurately they self-assess their competence. Students’ histories of privilege seem to influence how effectively they can initially do the kinds of metacognition conducive to furthering intellectual development when they enter college.

Sometimes a group’s hierarchy results from a lopsided division into some criterion-based majority/minority split. There, advantages, benefits, status, and even acceptance, deference, and respect often become inequitably and systematically conferred by identity on the majority group but not on the underrepresented minority groups.

Being a minority can invite being marked as “inferior,” with an unwarranted majority negative bias toward the minority, presuming the latter have inferior cognitive competence and even lower capacity for feeling than the majority. Perpetual exposure to such bias can influence the minority group to doubt themselves and unjustifiably underestimate their competence and capacity to perform. By employing paired measures, Wirth et al. (2021, p. 152 Figs.6.7 & 6.8) found recently that undergraduate women, who are the less represented binary gender in science, consistently underestimated their actual abilities relative to men (the majority) in science literacy.

We found that in the majority ethnic group (white Caucasians), both binary genders, on average, significantly outperformed their counterparts in the minority group (all other self-identified ethnicities combined) in both the competence scores of science literacy and the mean self-assessed competency ratings (Figure 1).

Figure 1. Graph of gender performance on measures of self-assessed competence ratings and demonstrated competence scores across ethnic majority/minority categories. This graph represents ten years of data collection of paired measures, but we only recently began to collect non-binary gender data within the last year, so this group is sparsely represented. Horizontal colored lines coded to the colored circles’ legend mark the positions of the means of scores and ratings in percent at the 95% confidence level.

Notably, in Figure 1, the non-binary gender groups, majority or minority, were the strongest academic group of the three gender categories based on SLCI scores. Still, relative to their performance, the non-binary groups felt that they performed less well than they actually did.

On a different SLCI dataset with a survey item on sexual preference rather than gender, researcher Kali Nicholas Moon (2018) found the same degree of diminished self-assessed competence relative to demonstrated competence for the small LGBT group (see Fig. 7 p. 24 of this link). Simply being a minority may predispose a group to doubt their competence, even if they “know their stuff” better than most.

These mean differences in performance shown in Figure 1 are immense. For perspective, pre-post measures in a GE college course or two in science rarely produce more than mean differences of more than a couple of percentage points on the SLCI. In both majority and minority groups, females, on average, underestimated their performance, whereas males overestimated theirs.

If a group receives constant messages that their thinking may be inferior, it is hardly surprising that they internalize feelings of inferiority that are damaging. Our evidence above from several groups verifies such a tendency. We showed that lower feelings of competence parallel significant deficit performance on a test of understanding science, an area relevant to achieving intellectual growth and meeting academic aspirations. Whether this signifies a general tendency of underconfidence in minority groups for meeting their aspirations in other areas remains undetermined.

Perpetuating privilege in higher education

Academe nurtures many hierarchies. Across institutions, “Best Colleges” rating lists perpetuate a myth that institutions that make the list are, in all ways, for all students “better than” those not on the list. Some state institutions actively promote a “flagship reputation,” implying the state’s other schools as “inferior.” Being in a community of peers that reinforces such hierarchical valuing confers damaging messaging of inferiority to those attending the “inferior” institutions, much as an ethnic majority casts negative messages to the minority.

Within institutions, different disciplines are valued differently, and people experience differential privileges across the departments and programs that focus on educating to support different disciplines. The degrees of consequences of stress, alienation, and physical endangerment are minor compared to those experienced by socially marginalized/minoritized groups. Nevertheless, advocating for any change in an established hierarchy in any community is perceived as disruptive by some and can provide consequences of diminished privilege. National communities of academic research often prove no exception.

Takeaways

Hierarchies usually define privilege, and the majority group often supports hierarchies detrimental to the well-being of minority groups. Although test scores are the prevalent measures used to measure learning mastery, paired measures of cognitive competence and self-assessed competence provide additional information about students’ affective feelings about content mastery and their developing capacity for accurate self-assessment. This information helps reveal the inequity across groups and monitors how well students can employ the higher education environment for advancing their understanding of specialty content and understanding of self. Paired measures confirm that groups of varied privilege fare differently in employing that environment for meeting their aspirations.

Understanding Bias in the Disciplines: Part 2 – the Physical and Quantitative Sciences

by Ed Nuhfer, California State University (Retired)
Eric Gaze, Bowdoin College
Paul Walter, St Edwards University
Simone Mcknight (Simone Erchov), Global Systems Technology

In Part 1, we summarized psychologists’ current understanding of bias. In Part 2, we connect conceptual reasoning and metacognition and show how bias challenges clear reasoning even in “objective” fields like science and math.

Science as conceptual

College catalogs’ explanations of general education (GE) requirements almost universally indicate that the desired learning outcome of the required introductory science course is to produce a conceptual understanding of the nature of science and how it operates. Focusing only on learning disciplinary content in GE courses squeezes out stakeholders’ awareness that a unifying outcome even exists.

Wherever a GE metadisciplinary requirement (for example, science) specifies a choice of a course from among the metadiscipline’s different content disciplines (for example, biology, chemistry, physics, geology), each course must communicate an understanding of the way of knowing established in the metadiscipline. That outcome is what the various content disciplines share in common. A student can then understand how different courses emphasizing different content can effectively teach the same GE outcome.

The guest editor led a team of ten investigators from four institutions and separate science disciplines (biology, chemistry, environmental science, geology, geography, and physics). Their original proposal was to investigate ways to improve the learning in the GE science courses. While articulating what they held in common as professing the metadiscipline of “science,” the investigators soon recognized that the GE courses they took as students had focused on disciplinary content but scarcely used that content to develop an understanding of science as a way of knowing. After confronting the issue of teaching with such a unifying emphasis, they later turned to the problem of assessing success in producing this different kind of understanding.

Upon discovering no suitable off-the-shelf assessment instrument to meet this need, they constructed the Science Literacy Concept Inventory (SLCI). This instrument later made possible this guest-edited series and the confirmation of knowledge surveys as valid assessments of student learning.

Concept inventories test understanding the concepts that are the supporting framework for larger overarching blocks of knowledge or thematic ways of thinking or doing. The SLCI tests nine concepts specific to science and three more related to the practice of science and connecting science’s way of knowing with contributions from other requisite GE metadisciplines.

Self-assessment’s essential role in becoming educated

Self-assessment is partly cognitive (the knowledge one has) and partly affective (what one feels about the sufficiency of that knowledge to address a present challenge). Self-assessment accuracy confirms how well a person can align both when confronting a challenge.

Developing good self-assessment accuracy begins with an awareness that having a deeper understanding starts to feel different from merely having surface knowledge needed to pass a multiple-choice test. The ability to accurately feel when deep learning has occurred reveals to the individual when sufficient preparation for a challenge has, in fact, been achieved. We can increase learners’ capacity for metacognition by requiring frequent self-assessments that give them the practice needed to develop self-assessment accuracy. No place needs teaching such metacognition more than the introductory GE courses.

Regarding our example of science, the 25 items on the SLCI that test understanding of the twelve concepts derive from actual cases and events in science. Their connection to bias lies in learning that when things go wrong when doing or learning science, some concept is unconsciously being ignored or violated. Violations are often traceable to bias that hijacked the ability to use available evidence.

We often say: “Metacognition is thinking about thinking.” When encountering science, we seek to teach students to “think about” (1) “What am I feeling that I want to be true and why do I have that feeling?” and (2) “When I encounter a scientific topic in popular media, can I articulate what concept of science’s way of knowing was involved in creating the knowledge addressed in the article?”

Examples of bias in physical science

“Misconceptions research” constitutes a block of science education scholarship. Schools do not teach the misconceptions. Instead, people develop preferred explanations for the physical world from conversations that mostly occur in pre-college years. One such explanation addresses why summers are warm and winters are cold. The explanation that Earth is closer to the sun in summer is common and acquired by hearing it as a child. The explanation is affectively comfortable because it is easy, with the ease coming from repeatedly using the neural network that contains the explanation to explain the seasonal temperatures we experience. We eventually come to believe that it is true. However, it is not true. It is a misconception.

When a misconception becomes ingrained in our brain neurology over many years of repeated use, we cannot easily break our habit of invoking the neural network that holds the misconception until we can bypass it by constructing a new network that holds the correct explanation. Still, the latter will not yield a network that is more comfortable to invoke until usage sufficiently ingrains it. Our bias tendency is to invoke the most ingrained explanation because doing so is easy.

Even when individuals learn better, they often revert to invoking the older, ingrained misconception. After physicists developed the Force Concept Inventory (FCI) to assess students’ understanding of conceptual relationships about force and motion, they discovered that GE physics courses only temporarily dislodged students’ misconceptions. Many students soon reverted to invoking their previous misconceptions. The same investigators revolutionized physics education by confirming that active learning instruction better promoted overcoming misconceptions than did traditional lecturing.

The pedagogy that succeeds seemingly activates a more extensive neural network (through interactive discussing, individual and team work on problem challenges, writing, visualizing through drawing, etc.) than was activated to initially install the misconception (learning it through a brief encounter).

Biases that add wanting to believe something as true or untrue are especially difficult to dislodge. An example of the power of bias with emotional attachment comes from geoscience.

Nearly all school children in America today are familiar with the plate tectonics model, moving continents, and ephemeral ocean basins. Yet, few realize that the central ideas of plate tectonics once were scorned as “Germanic pseudoscience” in the United States. That happened because a few prominent American geoscientists so much wanted to believe their established explanations as true that their affect hijacked these experts’ ability to perceive the evidence. These geoscientists also exercised enough influence in the U. S. to keep plate tectonics out of American introductory level textbooks. American universities introduced plate tectonics in introductory GE courses only years later than did Europe.

Example of Bias in Quantitative Reasoning

People usually cite mathematics as the most dispassionate discipline and the least likely for bias to corrupt. However, researchers Dan Kahan and colleagues demonstrated that bias also disrupts peoples’ ability to use quantitative data and think clearly.

Researchers asked participants to resolve whether a skin cream effectively treated a skin rash. Participants received data for subjects who did or did not use the skin cream. Among users, the rash got better in 223 cases and got worse in 75 cases. Of subjects who did not use the skin cream, the rash got better in 107 cases and worse in 21 cases.

Participants then used the data to select from two choices: (A) People who used the cream were more likely to get better or (B) People who used the cream were more likely to get worse. More than half of the participants (59%) selected the answer not supported by the data. This query was primarily a numeracy test in deducing the meaning of numbers.

Then, using the same numbers, the researchers added affective bait. They replaced the skin cream query with a query about the effects of gun control on crime in two cities. One city allowed concealed gun carry, and another banned concealed gun carry. Participants had to decide whether the data showed that concealed carry bans increased or decreased crime.

Self-identified conservative Republicans and liberal Democrats responded with a desire to believe acquired from their party affiliations. The result was even more erroneous than the skin cream case participants. Republicans greatly overestimated increased crime from gun bans, but no more than Democrats overestimated decreased crime from gun bans (Figure 1). When operating from “my-side” bias planted by either party, citizens significantly lost their ability to think critically and use numerical evidence. This was true whether the self-identified partisans had low or high numeracy skills.

Figure 1. Effect of bias on interpreting simple quantitative information (from Kahan et al. 2013, Fig. 8). Numerical data needed to answer whether a cream effectively treated a rash triggered low bias responses. When researchers employed the same data to determine whether gun control effectively changed crime, polarizing emotions triggered by partisanship significantly subverted the use of evidence toward what one wanted to believe.

Takeaway

Decisions and conclusions that appear based on solely objective data rarely are. Increasing metacognitive capacity produces awareness of the prevalence of bias.

Understanding Bias in the Disciplines: Part 1 – the Behavioral Sciences

by Simone Mcknight (Simone Erchov), Global Systems Technology
Ed Nuhfer, California State University (Retired)
Eric Gaze, Bowdoin College
Paul Walter, St Edwards University

Bias as conceptual

Bias arises from human brain mechanisms that process information in ways that make decision-making quicker and more efficient at the cognitive/neural level. Bias is an innate human survival mechanism, and we all employ it.

Bias is a widely known and commonly understood psychological construct. The common understanding of bias is “an inclination or predisposition for or against something.” People recognize bias by its outcome—the preference to accept specific explanations or attributions as true.

In everyday conversation, discussions about bias occur in preferences and notions people have on various topics. For example, people know that biases may influence the development of prejudice (e.g., ageism, sexism, racism, tribalism, nationalism), political, or religious beliefs.

A deeper look reveals that some of these preferences are unconscious. Nevertheless, they derive from a related process called cognitive bias, a propensity to use preferential reasoning to assess objective data in a biased way. This entry introduces the concept of bias, provides an example from the behavioral sciences, and explains why metacognition can be a valuable tool to counteract bias. In Part 2, which follows this entry, we provide further examples from hard science, field science, and mathematics.

Where bias comes from

Biases develop from the mechanisms by which the human brain processes information as efficiently as possible. These unconscious and automatic mechanisms make decision-making more efficient at the cognitive/neural level. Most mechanisms that help the human brain make fast decisions are credited to adaptive survival. Like other survival mechanisms, bias loses value and can be a detriment in a modern civilized world where threats to our survival are infrequent challenges. Cognitive biases are subconscious errors in thinking that lead to misinterpreting future information from the environment. These errors, in turn, impact the rationality and accuracy of decisions and judgments.

When we frame unconscious bias within the context of cognitive bias and survival, it is easier to understand how all of us have inclinations to employ bias and why any discipline that humans manage is subject to bias. Knowing this makes it easier to account for the frequent biases affecting the understanding and interpreting of diverse kinds of data.

People easily believe that bias only exists in “subjective” disciplines or contexts where opinions and beliefs seem to guide decisions and behavior. However, bias manifests in how humans process information at the cognitive level. Although it is easier to understand bias as a subjective tendency, the typical way we process information means that bias can pervade all of our cognition.

Intuitively, disciplines relying on tangible evidence, logical arguments, and natural laws of the physical universe would seem factually based and less influenced by feelings and opinion. After all, “objective disciplines” do not predicate their findings on beliefs about what “should be.” Instead, they measure tangible entities and gather data. However, even in the “hard science” disciplines, the development of a research question, the data collected, and the interpretations of data are vulnerable to bias. Tangible entities such as matter and energy are subject to biases as simple as differences in perception of the measured readings on the same instrument. In the behavioral sciences, where investigative findings are not constrained by natural law, bias can be even harder to detect. Thus, all scientists carry bias into their practice of science, and students carry bias into their learning of it.

Metacognition can help counter our tendencies toward bias because it involves bringing relevant information about a process (e.g., conducting research, learning, or teaching) into awareness and then using that awareness to guide subsequent behaviors.

Consequences of bias

Bias impacts individual understanding of the world, the self, and how the self navigates the world – our schemas. These perceptions may impact elements of identity or characterological elements that influence the likelihood of behaving in one way versus another.

Bias should be assumed as a potentially influential factor in any human endeavor. Sometimes bias develops for an explanation after hearing it in childhood and then invoking that explanation for years. Even after seeing the evidence against that bias, our initial explanations are difficult to replace with ones better supported by evidence because we remain anchored to that initial knowledge. Adding a personal emotional attachment to an erroneous explanation makes replacing it even more difficult. Scientists can have emotional attachments to particular explanations of phenomena, especially their own explanations. Then, it becomes easy to selectively block out or undervalue evidence that modifies or contradicts the favored explanation (also known as confirmation bias).

Self-assessment, an example of long-standing bias in behavioral science

As noted in the introduction, this blog series focuses on our team’s work related to self-assessment. Our findings countered results from scores of researchers who replicated and verified the testing done in a seminal paper by Kruger and Dunning (1999). Their research asserted that most people were overconfident about their abilities, and the least competent people had the most overly optimistic perceptions of their competence. Researchers later named the phenomenon the “Dunning-Kruger effect,” and the public frequently deployed “the effect” as a label to disparage targeted groups as incompetent. “The effect” held attraction because it seemed logical that people who lacked competence also lacked the skills needed to recognize their deficits. Quite simply, people wanted to believe it, and replication created a consensus with high confidence in concluding that people, in general, cannot accurately self-assess.

While a few researchers did warn about likely weaknesses in the seminal paper, most behavioral scientists selectively ignored the warnings and repeatedly employed the original methodology. This trend of replication continued in peer-reviewed behavioral science publications through at least 2021.

Fortunately, the robust information storage and retrieval system that characterizes the metadiscipline of science (which is a characteristic distinguishing science from technology as ways of knowing) makes it possible to challenge a bias established in one discipline by researchers from another. Through publications and open-access databases, the arguments that challenge an established bias then become available. In this case, the validity of “the effect” resided mainly in mathematical arguments and not, as presumed, arguments that resided solely within the expertise of behavioral scientists.

No mathematics journal had ever hosted arguments addressing the numeracy of arguments that established and perpetuated the belief in “the effect.” However, mathematics journals offered the benefit of reviewers who specialized in quantitative reasoning and were not emotionally attached to any consensus established in behavioral science journals. These reviewers agreed that the long-standing arguments for supporting the Dunning-Kruger effect were mathematically flawed.

In 2016 and 2017, Numeracy published two articles from our group that detailed the mathematical arguments that established the Dunning-Kruger effect conclusions and why these arguments are untenable. When examined by methods the mathematics reviewers verified as valid, our data indicated that people were generally good at self-assessing their competence and confirmed that there were no marked tendencies toward overconfidence. Experts and novices proved as likely to underestimate their abilities as to overestimate them. Further, the percentage of those who egregiously overestimated their abilities was small, in the range of about 5% to 6% of participants. However, our findings confirmed a vital conclusion of Kruger and Dunning (1999): experts self-assess better than novices (variance decreases as expertise increases), and self-assessment accuracy is attainable through training and practice.

By 2021, the information released in Numeracy began to penetrate the behavioral science journals. This blog series, our earlier posts on this site, and archived presentations to various audiences (e.g., the National Numeracy Network, the Geological Society of America) further broadened awareness of our findings.

Interim takeaways

Humans construct their learning from mentally processing life experiences. During such processing, we simultaneously construct some misconceptions and biases. The habit of drawing on a misconception or bias to explain phenomena ingrains it and makes it difficult to replace with correct reasoning. Affective attachments to any bias make overcoming the bias extremely challenging, even for the most accomplished scholars.

It is essential to realize that we can reduce bias by employing metacognition to recognize bias originating from within us at the individual level and by considering bias that influences us but is originated from or encouraged by groups. In the case above, we were able to explain the bias within the Behavioral Sciences disciplines by showing how repeatedly mistaking mathematical artifacts as products of human behavior produced a consensus that held understanding self-assessment captive for over two decades.

Metacognitive self-assessment seems necessary for initially knowing self and later for recognizing one’s own personal biases. Self-assessment accuracy is valuable in using available evidence well and reducing the opportunity for bias to hijack our ability to reason. Developing better self-assessment accuracy appears to be a very worthy objective of becoming educated.