Micro-Metacognition Makes It Manageable

By Dr. Lauren Scharff, U. S. Air Force Academy *

For many of us, this time of year marks an academic semester’s end. It’s also intertwined with a major holiday season. There is so much going on – pulling our thoughts and actions in a dozen different directions. It almost seems impossible to be metacognitive about what we’re doing while we’re doing it: grading that last stack of exams or papers, finalizing grades, catching up on all those work items that have been on the back burner but need to get done before the semester ends. And that’s just a slice of our professional lives. Add in all the personal tasks and aspirations for these final few weeks of the year, and it’s Go, Go, Go until the day is over.

Well, that was a bit cathartic to get out, but also a bit depressing. Logically I know that by taking the time to reflect and use that awareness to help guide my behaviors (i.e. engage in metacognition), I will feel energized and revitalized because I’ll have a plan with a good foundation. I will more likely be successful in whatever it is I’m hoping to accomplish, especially if I regularly take the time to reflect and fine tune my efforts. But the challenge is, how do I fit it all in?

My proposed solution is micro-metacognition!

So, what do I mean by that? I think micro-metacognition is analogous to taking the stairs whenever you can rather than signing up for a new gym membership. Stairs are readily available at no cost and can be used spur-of-the-moment. In comparison, the gym membership requires a more concerted effort and larger chunks of time to get to the facility, work out, clean up and head home. In the more academic realm, micro metacognition falls in line with the spirit of James Lang’s (2016) Small Teaching recommendations. He advocates for the powerful impact of even small changes in our teaching (e.g. how we use the first 5 minutes of class). In other words, we don’t have to completely redesign a course or our way of teaching to see large benefits.

To help place micro-metacognition into context, I will borrow a framework from Poole and Simmons (2013), who suggested a “4M” model for conceptualizing the level of impact of SoTL work: micro (individual/classroom), meso (department/ program), macro (institutional), and mega (beyond a single institution). In this case though, we’re looking at engagement in metacognitive practices, so the entity or level of focus will always be the individual, and the scale will refer to the amount of planning, effort, and time needed for the metacognitive practice. This post focuses on instructors being metacognitive about their practice of teaching, but I believe that parallels can easily be made for students’ engagement in metacognition as they are learning.

The 4M Metacognition Framework

Micro-metacognition – Use of isolated, low-cost tactics to promote metacognitive instruction when engaged in single tasks (e.g. grading a specific assignment – see below for fleshed out example). These can be used without investments in advance planning.

Meso-metacognition – Use of tactics to promote metacognitive instruction throughout an individual lesson or when incorporating a specific type of activity (e.g. discussion or small group work) across multiple lessons. These tactics have been given more forethought with respect to integration with lesson / activity objectives.

Macro-metacognition – Use of more regular tactics to promote metacognitive instruction across an entire course / semester. Planning for these would be more long-term and closely integrated with learning objectives for the course or with professional development goals of the instructor. (For an example of this level of effort, see Use of a Guided Journal to Support Development of Metacognitive Instructors.)

Mega-metacognition – Use of tactics to promote metacognitive instruction across an instructor’s entire set of courses and professional activities (and beyond). At this level of engagement, metacognition will likely be a “way of doing things” for the instructor, but each new engagement will still require conscious effort and planning to support goals and objectives.

An Example of Micro Metacognition

Micro-metacognition efforts are not pre-planned when the instructional task is planned; they are added later as the idea crosses the instructor’s mind and opportunity arises.

For example, when I am about to start grading a specific group of papers, I might reflect that in addition to the formally-stated learning objectives that will be assessed on the rubric, I want to support growth mindset in my students for their future writing efforts. This additional goal could come about from a recent reading on mindset or discussion with my colleagues. I know that I would be likely to forget this goal when I’m focused on the other rubric aspects of the grading. So, I write that goal on a stickie note and put it where I am likely to see it when grading. Then, when I am grading, I have an easy-to-implement awareness aide to add comments in the papers that might specifically support my students’ growth mindset.

Image showing an office with a sticky note stuck to the corner of a computer screen. The note says "Promote Growth Mindset -- encourage exploration of new ideas & connections"

In sum: easily implemented stickie note –> promotes awareness of goal –> self-regulation of desired grading behavior on that specific instructional task == Micro-metacognitive Instruction!

I can think of lots of other ways instructors might incorporate micro-metacognition in their instructional endeavors, from the proverbial string tied to one’s finger, to pop-up calendar prompts, to asking a student for a reminder to attend to questions when we get to a particular topic. Or, awareness might come without an intentional prompt. The key is to then use that awareness to self-regulate some aspect of our instructional behavior in support of student learning and development. The opportunities are endless!

I hope you are motivated as you enter the new year. Happy holidays!

——————-

Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning.

Poole, G., & Simmons, N. (2013). The contributions of the scholarship of teaching and learning to quality enhancement in Canada. In G. Gordon, & R. Land (Eds.). Quality enhancement in higher education: International perspectives (pp. 118-128). London: Routledge.

* Disclaimer: The views expressed in this document are those of the author and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.

Metacognitive Self-Assessment, Competence and Privilege

by Steven Fleisher, Ph.D., California State University Channel Islands

Recently I had students in several of my classes take the Science Literacy Concept Inventory (SLCI) including self-assessment (Nuhfer, et al., 2017). Science literacy addresses one’s understanding of science as a way of knowing about the physical world. This science literacy instrument also includes self-assessment measures that run parallel with the actual competency measures. Self-assessment skills are some of the most important of the metacognitive competencies. Since metacognition involves “thinking about thinking,” the question soon becomes, “but thinking about what?”

Dunlosky and Metcalfe (2009) framed the processes of metacognition across metacognitive knowledge, monitoring, and control. Metacognitive knowledge involves understanding how learning works and how to improve it. Monitoring involves self-assessment of one’s understanding, and control then involves any needed self-regulation. Self-assessment sits at the heart of metacognitive processes since it sets up and facilitates an internal conversation in the learner, for example “Am I understanding this material at the level of competency needed for my upcoming challenge?” This type of monitoring then positions the learner for any needed control or self-regulation, for instance “Do I need a change my focus, or maybe my learning strategy?” Further, self-assessment is affective in nature and is central to how learning works. From a biological perspective, learning involves the building and stabilizing of cognitive as well as affective neural networks. In other words, we not only learn about “stuff”, but if we engage our metacognition (specifically self-assessment in this instance), we are enhancing our learning to include knowing about “self” in relation to knowing about the material.

This Improve with Metacognition posting provides information that was shared with my students to help them see the value of self-assessing and for understanding its relationship with their developing competencies and issues of privilege. Privilege here is defined by factors that influence (advantage or disadvantage) aggregate measures of competence and self-assessment accuracy (Watson, et al., 2019). Those factors involved: (a) whether students were first-generation college students, (b) whether they were non-native English-language students, and (c) whether they had an interest in science.

The figures and tables below result from an analysis of approximately 170 students from my classes. The narrative addresses the relevance of each of the images.

Figure 1 shows the correlation between students’ actual SLCI scores and their self-assessment scores using Knowledge Survey items for each of the SLCI items (KSSLCI). This figure was used to show students that their self-assessments were indeed related to their developing competencies. In Figure 2, students could see how their results on the individual SLCI and KSSLCI items were tracking even more closely than in Figure 1, indicating a fairly strong relationship between their self-assessment scores and actual scores.

scatterplot graph of knowledge survey compared to SCLI scores — Figure 1. Correlation with best-fit line between actual competence measures via a Science Literacy Concept Inventory or SLCI (abscissa) and self-assessed ratings of competence (ordinate) via a knowledge survey of the inventory (KSSLCI) wherein students rate their competence to answer each of the 25 items on the inventory prior to taking the actual test.

scatter plot of SCLI scores and knowledge survey scores by question — Figure 2. Correlation with best-fit line between the group of all my students’ mean competence measures on each item of the Science Literacy Concept Inventory (abscissa) and their self-assessed ratings of competence on each item of the knowledge survey of the inventory (KSSLCI).

Figure 3 demonstrates the differences in science literacy scores and self-assessment scores among their different groups as defined by the number of science courses taken. Students could readily see the relationship between the number of science courses taken and improvement in science literacy. More importantly in this context, students could see that these groups had a significant sense of whether or not they knew the information, as indicated by the close overlapping of each pair of green and red diamonds. Students learn that larger numbers of participants can provide more confidence to where the true means actually lies. Also, I can show the meaning of variation differences within and between groups. In answering questions about how we know that more data would clarify relationships, I bring up an equivalent figure from our national database that shows the locations of the means within 99.9% confidence and the tight relationship between groups’ self-assessed competence and their demonstrated competence.

categorical plot by number of college science courses completed — Figure 3. Categorical plot of my students in five class sections grouped by their self-identified categories of how many college-level science courses that they have actually completed. Revealed here are the groups’ mean SLCI scores and their mean self-assessed ratings. Height of the green (SLCI scores) and red (KSSLCI self-assessments) diamonds reveals with 95% confidence that the actual mean lies within these vertical bounds.

Regarding Figure 4, it is always fun to show students that there’s no significant difference between males and females in science literacy competency. This information comes from the SLCI national database and is based on over 24,000 participants.

categorical plot by binary gender — Figure 4. Categorical plot from our large national database by self-identified binary gender categories shows no significant difference by gender in competence of understanding science as a way of knowing.

It is then interesting to show students in that, in their smaller sample (Figure 5), there is a difference between the science literacy scores of males and females. The perplexed looks on their faces are then addressed by the additional demographic data in Table 1 below.

categorical plot by binary gender for individual class — *Figure 5. Categorical plot of just my students by binary gender reveals a marginal difference between females and males, rather than the gender-neutral result shown in Fig. 4.*

In Table 1, students could see that higher science literacy scores for males in their group were not due to gender, but rather, were due to significantly higher numbers of English as a non-native language for females. In other words, the women in their group were certainly not less intelligent, but had substantial, additional challenges on their plates.

Table 1: percentages of male and female students as first generation, English and non-native speaker, and with respect to self-report interest to major in science

Students then become interested in discovering that the women demonstrated greater self-assessment accuracy than did the men, who tended to overestimate (Figure 6). I like to add here, “that’s why guys don’t ask for directions.” I can get away with saying that since I’m a guy. But more seriously, I point out that rather than simply saying women need to improve in their science learning, we might also want to help men improve in their self-assessment accuracy.

categorical plot by gender including self-assessment data — Figure 6. The categorical plot of SLCI scores (green diamonds) shown in Fig. 5 now adds the self-assessment data (red diamonds) of females and males. The trait of females to more accurately self-assess that appears in our class sample is also shown in our national data. Even small samples taken from our classrooms can yield surprising information.

In Figure 7, students could see there was a strong difference in science literacy scores between Caucasians and Hispanics in my classes. The information in Table 2 below was then essential for them to see. Explaining this ethnicity difference offers a wonderful discussion opportunity for students to understand not only the data but what it reveals is going on with others inside their classrooms.

*Figure 7. The categorical plot of SLCI scores by the two dominant ethnicities in my classroom. My campus is a Hispanic Serving Institution (HSI). The differences shown are statistically significant.*

Table 2 showed that the higher science literacy scores in this sample were not simply due to ethnicity but were impacted by significantly greater numbers of first-generation students and English as a non-native language between groups. These students are not dumb but do not have the benefits in this context of having had a history of education speak in their homes and are navigating issues of English language learning.

Table 2: percentage of white and hispanic students who report to be first generation students, English as non-native speakers, and interested in majoring in science.

When shown Figure 8, which includes self-assessment scores as well as SLCI scores, students were interested to see that both groups demonstrated fairly accurate self-assessment skills, but that Hispanics had even greater self-assessment accuracy than their Caucasian colleagues. Watson et. al (2019) noted that strong self-assessment accuracy for minority groups comes about from a need for being understandably cautious.

categorical plot by ethnicity and including self-assessment — Figure 8. The categorical plot of SLCI scores and self-assessed competence ratings for the two dominant ethnicities in my classroom. Groups’ collective feelings of competence, on average, are close to their actual competence. Explaining these results offered a wonderful discussion opportunity for students.

Figure 9 shows students that self-assessment is real. In seeing that most of their peers fall within an adequate range of self-assessment accuracy (between +/- 20 percentage points), students begin to see the value of putting effort into developing their own self-assessment skills. In general, results from this group of my students are similar to those we get from our larger national database (See our earlier blog post, Paired Self-Assessment—Competence Measures of Academic Ranks Offer a Unique Assessment of Education.)

distribution of self-assessment accuracy for individual course — Figure 9. The distribution of self-assessment accuracy of my students in percentage points (ppts) as measured by individuals’ differences between their self-assessed competence by knowledge survey and their actual competence on the Concept inventory.

Figure 10 below gave me the opportunity to show students the relationship between their predicted item-by-item self-assessment scores (Figure 9) and their postdicted global self-assessment scores. Most of the scores fall between +/- 20 percentage points, indicating good to adequate self-assessment. In other words, once students know what a challenge involves, they are pretty good at self-assessing their competency.

distribution of self-assessment accuracy for individual course after taking SCLI — Figure 10. The distribution of self-assessment accuracy of my students in percentage points (ppts) as measured by individuals’ differences between their postdicted ratings of competence after taking the SLCI and their actual scores of competence on the Inventory. In general, my students’ results are similar in self-assessment measured in both ways.

In order to help students further develop their self-assessment skills and awareness, I encourage them to write down how they feel they did on tests and papers before turning them in (postdicted global self-assessment). Then they can compare their predictions with their actual results in order to fine-tune their internal self-assessment radars. I find that an excellent class discussion question is “Can students self-assess their competence?” Afterward, reviewing the above graphics and results becomes especially relevant. We also review self-assessment as a core metacognitive skill that ties to an understanding of learning and how to improve it, the development of self-efficacy, and how to monitor their developing competencies and control their cognitive strategies.

References

Dunlosky, J. & Metcalfe, J. (2009). Metacognition. Sage Publications Inc., Thousand Oaks, CA.

Nuhfer, E., Fleisher, S., Cogan, C., Wirth, K., & Gaze, E. (2017). How Random Noise and a Graphical Convention Subverted Behavioral Scientists’ Explanations of Self-Assessment Data: Numeracy Underlies Better Alternatives. Numeracy, Vol 10, Issue 1, Article 4. DOI: http://dx.doi.org/10.5038/1936-4660.10.1.4

Watson, R., Nuhfer, E., Nicholas Moon, K., Fleisher, S., Walter, P., Wirth, K., Cogan, C., Wangeline, A., & Gaze, E. (2019). Paired Measures of Competence and Confidence Illuminate Impacts of Privilege on College Students. Numeracy, Vol 12, Issue 2, Article 2. DOI: https://doi.org/10.5038/1936-4660.12.2.2

The Metacognitive Reading Strategies Questionnaire (MRSQ): Cross-Cultural Comparisons

by Roman Taraban, Ph.D. Texas Tech University

When you read, do you ask yourself whether the material is contributing to your knowledge of the subject, whether you should revise your prior knowledge, or how you might use the new knowledge that you are acquiring? Do you highlight information or make notes in the margins to better remember and find information later on? Prior research by Pressley and colleagues (e.g., Pressley & Afflerbach, 1995) suggested that the type of metacognitions suggested by reading strategies like these were critical for effective reading comprehension.

photo of a stack of books with a pair of reading glasses on top

Inspired by that research, Taraban et al. (2000) conducted a study involving 340 undergraduates and 35 reading strategies like those suggested by Pressley and colleagues and found that self-reports of strategy use were significantly associated with grade-point averages (GPA). Specifically, students who reported higher use of reading strategies also had higher GPAs. Additionally, responses to open-ended questions showed that students who could name more reading strategies and reading goals also had significantly higher GPAs.

The data in Taraban et al. (2000) overwhelmingly suggested a strong positive relationship between students’ knowledge and use of reading goals and strategies and their academic performance. More generally, data by Taraban et al. and others suggest that effective reading depends on metacognitive processing – i.e., on directed cognitive effort to guide and regulate comprehension. Skilled readers know multiple strategies and when to apply them. In the remainder of this post, I review subsequent developments associated with metacognitive reading strategies, including cross-cultural comparisons, as well as raising a question about the relevance of these strategies to present-day text processing and comprehension given widespread technological developments.

Analytic VS Pragmatic Reading Strategies

In 2004, my students and I created a questionnaire, dubbed the Metacognitive Reading Strategies Questionnaire (MRSQ) (Taraban et al., 2004). The questionnaire drew on the strategies tested earlier in Taraban et al. (2000) and organized the strategies into two subscales through factor analytic methods: analytic strategies and pragmatic strategies. The analytic scale relates to cognitive strategies like making inferences and evaluating the text (e.g., After I read the text, I consider other possible interpretations to determine whether I understood the text.). The pragmatic scale relates to practical methods for finding and remembering information from the text (e.g., I try to underline when reading in order to remember the information.). Students respond to these statements using a five-point Likert-type scale: Never Use, Rarely Use, Sometimes Use, Often Use, Always Use.

Initial applications of the MRSQ suggested that the two-factor model could aid in better understanding students’ use of metacognitive comprehension strategies. Specifically, in students’ self-reports of expected GPA for the coming academic year, there was a significant positive correlation with analytic strategies but a non-significant correlation with pragmatic strategies, which suggested that students who reported higher use of analytic strategies also anticipated doing well academically in the coming academic year.

Cross-Cultural Explorations of Reading Strategies

Vianty (2007) used the MRSQ to explore difference in students’ use of metacognitive reading strategies in their native language, Bahasa Indonesia, and their second language, English. Participants were students in a teacher education program who completed the MRSQ in English and Bahasa Indonesia. Vianty found that students processed language differently in their native language compared to a non-native language.

In comparing mean use of analytic strategies when reading in their native language compared to English, Vianty found that nearly all means were higher for Bahasa Indonesia. T-tests showed significant differences favoring Bahasa Indonesia for eight out of sixteen analytic strategies. Conversely, four of the six pragmatic strategies were favored when reading English, however, only one difference (I take notes when reading in order to remember the information) was significant on a t-test. Vianty concluded that students used analytic strategies significantly more in Bahasa Indonesia than English. Conversely, use of pragmatic strategies was higher when reading in English, but the effect was weak.

Taraban et al. (2013) compared US and Indian engineering undergraduates on their application of analytic and pragmatic strategies. The language of instruction in Indian universities is English; however, this is not typically the native language (the mother tongue) of the students. Therefore, the researchers predicted lower use of analytic strategies and higher use of pragmatic strategies among Indian students compared to US students, reasoning from the findings in Vianty (2007). The latter but not former prediction was supported. Indeed, Indian students applied analytic strategies significantly more frequently than US students. Pragmatic strategy use was significantly lower than analytic strategy use for US students but not for Indian students, who applied analytic and pragmatic strategies equally often. Contrary to the findings in Vianty (2007), these findings suggest that students can make significant use of analytic and pragmatic strategies in a non-native language.

The most comprehensive cross-linguistic comparison was conducted recently by Gavora et al. (2019), who compared analytic and pragmatic strategy use, measured by variants of the MRSQ, among 2692 students from Poland, Hungary, Slovakia, and the Czech Republic, enrolled in education programs, primarily teacher and counseling. Students in Hungary, Slovakia, and the Czech Republic reported significantly higher use of pragmatic over analytic strategies. Students in Poland showed a converse preference, reporting significantly more frequent use of analytic strategies. Quite striking in the results were the significant correlations between pragmatic strategy use and GPA, and analytic strategy use and GPA, for all four countries. Specifically, the correlation showed that higher frequency use of both pragmatic and analytic strategies was associated with more successful academic performance.

Gavora et al. (2019) suggest that “In order to succeed academically, students direct their reading processes not towards comprehension but to remembering information, which is the core component of the pragmatic strategy” (p. 12). Their recommendation, that “educators’ attention should be focused on developing especially analytic strategies in students,” is strongly reminiscent of the ardor with which Pressley and colleagues began promoting metacognitive reading strategies beginning in the elementary grades.

However, given the significant correlations between both analytic and pragmatic strategy use with GPA, it may be that the predominance of analytic strategies is not what is important, but whether application of either type of strategy – analytic or pragmatic – aids students in their academic achievement. The data from Vianty (2007) may be informative in this regard, specifically, the finding that those students applied pragmatic strategies more frequently than analytic strategies when the context – reading outside their native language – dictated a more pragmatic approach to reading and comprehension.

A relevant point made by Gavora et al. relates to the samples that have been tested to-date, and the relevance of context to strategy use. They point out that in contexts like engineering (e.g., Taraban et al., (2013), the context may support more analytic thinking and analytic strategy use. The Gavora et al., sample consisted of humanities students, which, on their argument, may have resulted in an overwhelming affirmation of pragmatic strategies. Further comparisons across students in different programs is certainly warranted.

Changing Times: The Possible Influence of Technology on Reading

An additional question comes to mind, which is the effect of widespread technology in instructional settings. When I, like others, am uncertain about a definition, algorithm, theory, etc., I find it very easy to simply Google the point or look for a YouTube, which I simply need to read or watch for an explanation. This personal observation suggests that perhaps the strategies that are probed in the MRSQ may, at this point, be incomplete, and in some instances, somewhat irrelevant. The next step should be to ask current students what strategies they use to aid comprehension. Their responses may lead to new insights into contemporary student metacognitions that assist them in learning.

In conclusion, there is no doubt that metacognitive strategies are essential to effective information processing. However, there may be room to reconsider and update the strategies that students employ when reasoning and searching for information and insights to guide and expand comprehension and learning. It may be that current technology has made students more pragmatic and a promising goal for further research would be to uncover the ways in which that pragmatism is being expressed through new search strategies.

References

Gavora, P., Vaculíková, J., Kalenda, J., Kálmán, O., Gombos, P., Świgost, M., & Bontová, A. (2019). Comparing metacognitive reading strategies among university students from Poland, Hungary, Slovakia and the Czech Republic. Journal of Further and Higher Education, 1-15.

Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.

Taraban R, Kerr M, Rynearson K (2004) Analytic and pragmatic factors in college students’ metacognitive reading str ategies. Reading Psychology, 25(2), 67–81.

Taraban, R., Rynearson, K., & Kerr, M. (2000). College students’ academic performance and self-reports of comprehension strategy use. Reading Psychology, 21, 283–308.

Taraban, R., Suar, D., & Oliver, K. (2013). Information literacy of US and Indian engineering undergraduates. SpringerPlus, 2(1), 244.

Vianty, M. (2007). The comparison of students’ use of metacognitive reading strategies between reading in Bahasa Indonesia and in English. International Education Journal,8(2), 449–460.

From Faculty to Chair: Lessons Learned

by Dr. Scott Santos, faculty member who became Chair of the Department of Biological Sciences in 2018; thus, he was thrust into leadership of the curricular redesign project.

In the final post of “The Evolution of M etacognition in Biological Sciences” guest series, Dr. Scott Santos shares his experience of moving from a faculty member in the department when the process of improving metacognition in the department began, to becoming chair and suddenly in a position to lead it. He also shares key lessons we learned over the course of the project and looks ahead to what’s next for the Biology department.

Learning about Metacognition

“Metacognition…Huh? What’s that?!” is what popped to mind the first time I came across that word in an email announcing that our department would be investigating ways to integrate it into courses at our 2017 Auburn University (AU) Department of Biological Sciences (DBS) Annual Faculty Retreat. Testimony to my naïveté on metacognition at the time comes from the fact that the particular email announcing the above is the first containing that specific word among 100,000+ correspondences dating back to 2004 when I started as a faculty member.

The email mentioning metacognition prompted me to spend a few minutes researching the word and, not surprisingly, discovering a wealth of internet resources. One of the most useful I found among these came from the Center for Teaching at fellow Southeastern Conference school, Vanderbilt University, where it was defined as “…. simply, thinking about one’s thinking” (Chick, 2015). I found this interesting since it reminded me of a recurring comment I have heard over the years amongst individuals who have successfully defended their Ph.D. dissertations, namely that one’s defense makes you realize how much you know about one particular area of knowledge while realizing how little you know about everything else.

The point that jumped out at me concerning this potential analogy was that, if it represented a genuine example of metacognition, it evolves in an individual over multiple years as they experience the trials and tribulations (as well as rewards and eventual success) associated with obtaining a terminal degree. Ambitiously, we were taking on the challenge of attempting to instill in early-career students an awareness and recognition of their strengths and weaknesses across the spectrum of learning, writing, reading, etc. As you can imagine, this was my first indicator that we had some work to do. 

Educating Our Department

So how did the DBS faculty at AU approach this seemingly daunting task of bringing metacognition “to the masses”?

Firstly, our previous departmental leadership had the foresight to start the process by having the retreat facilitated by highly-qualified individuals. Specifically, Dr. Ellen Goldey (currently Dean, Wilkes Honors College, Florida Atlantic University; formerly Department Chair, Wofford College, SC) and Dr. April Hill (Chair, Department of Biology, University of Richmond) were recruited as two nationally-recognized leaders involved in the National Science Foundation (NSF)-funded Vision and Change (V&C) Report (Brewer & Smith 2011) to conduct a workshop that included integration of metacognition into our curricula. This proved highly useful to having our faculty begin to wrap our collective minds around what metacognition was (and could be) along with how we might begin approaching its integration into our existing and future courses. This has been followed up by general faculty meetings, and subcommittee meetings such as those of the DBS Curriculum Committee, which have occurred at regular intervals to undertake this process.

photo of Bio Dept faculty fall 2019 — Fall 2019 Biology faculty retreat group photo. Scott Santos, front and center, giving the Shaka sign.

Overall, I am happy to report that we have made some significant progress in this area, including holding specialized workshops on the topic and discussing approaches to incorporate metacognitive prompts into midterms, finals, and surveys of undergraduate student research experiences that collect responses for future qualitative analyses, for integration of metacognition development and assessment into our budding ePortfolio initiative, and other activities (like this blog series). However, these modest successes have not come without challenges: as our “metacognition massacre” experience taught us, it takes significant levels of time and energy for such efforts to come to fruition and to seed and foster support for these efforts among the faculty charged with bringing metacognition into the classroom. 

Key Insights

It has now been several years since AU DBS started our initiatives with metacognition, and during this time I have transitioned from an individual faculty member “in the trenches” to Chair of the department and thus charged with “leading the troops.” While I would be well-off financially if I had received a nickel for every time I have been offered “congratulation, and condolences” in the year and a half since becoming Chair, it has given me a new and different perspective on our metacognitive efforts:

First, a passionate and dedicated team is needed for initiatives like this to prosper and we, as a department, have been fortunate to have that in the form of our AU colleagues who are also contributing blogs in this series. Importantly, they belong to multiple units outside DBS, thus bringing the needed expertise and perspective that we lacked or might miss, respectively. We are greatly indebted to them, and departmental chairs and heads interested or intending to start similar initiatives would be wise to establish and cultivate such collaborations early in the process. It is very helpful to have expert advice when tackling issues that are unfamiliar to most of the faculty.
Second, working with your faculty on understanding what metacognition is, along with defining expectations and assessment for initiatives around it, are paramount for your department’s immediate success with implementing activities from such efforts. In our case, the fact that many AU DBS faculty wrestled with the concept of metacognition meant that we had to invest more time for calibration before discussion could move forward.
Third, the significance of solicitating undergraduate student participation during the development and implementation stages of the process should not be undervalued since they are the constituents who our efforts are ultimately targeting and thus deserve a voice at the table. Although our posts in this series have highlighted inflection points for the faculty as we moved our curriculum toward more metacognition, it is critical to note that we involved students as partners throughout the process. Some strategies we used include organizing student focus groups led by facilitators outside the department, conducting surveys, and inviting students to some meetings and department retreats.

Importantly, this should not be considered an exhaustive list and instead should serve as a general guide of issues to consider from someone who has had an opportunity to both witness and participate in the process from the departmental faculty and leadership perspectives. 

Looking Toward the Future

What does the future hold for AU DBS when it comes to metacognition? On one hand, we will continue in the short-term to implement the initiatives described above while being opportunistic in improving them, which we consider to be a strategy consistent with the current stage of our efforts to develop metacognitive abilities in students enrolled in our programs. On the other hand, the long-term forecast, at least from the departmental standpoint, is more amorphous, with reasons for this including our need to involve a large number of newly recruited faculty. We look forward to new directions and possibilities as we learn about new strategies from our colleagues, though we recognize the need to balance and maintain synergy between departmental undergraduate and graduate programs in the face of limited resources.

Finally, a key element for metacognition highlighted by Vanderbilt’s Center of Learning is “recognizing the limit of one’s knowledge or ability and then figuring out how to expand that knowledge or extend the ability.” Given that, I would like to think that Auburn’s Department of Biological Sciences itself is attempting to be metacognitive in its approach to preparing and fostering metacognition in students, and it will be interesting to see how our current efforts evolve in the future. 

References:

Chick, N. (2015). Metacognition: Thinking about one’s thinking. Vanderbilt University-The Centre for Teaching.

Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science, Washington, DC.

Assessing Metacognition: A Plan for Learning Improvement

In the fourth post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Lindsay Doukopoulos describes the goals and outcomes of the spring meetings of the Biology curriculum committee that were led by Biggio Center and the Office of Academic Assessment. Namely, they sought to create a questionnaire that would be given to all graduating students that would allow them reflect on their learning over the course of their academic career and to create a rubric to measure the quality of metacognition in their responses.

by Dr. Lindsay Doukopoulos, Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning. In collaborating with the Office of Academic Assessment on the Learning Improvement Initiative, she leads faculty development initiatives designed to connect faculty with strategies, resources, and partners to support their teaching.

In Spring of 2019, Katie Boyd and I led four meetings with Biology’s curriculum committee with the two-part goal of producing a set of metacognitive-reflection questions to be completed by every graduating student in their capstone course and a rubric to assess the quality of metacognition evidenced in the reflections.

Developing the Rubric

In the first workshop, our goal was to help faculty unpack the definition of metacognition into categories and decide how many levels or standards to maintain within the rubric. In other words, we were hoping to fill in the x and y axes of the Metacognition rubric.

To facilitate this discussion, we brought two rubrics designed to measure metacognition. One came from the General Learning Outcome site of Cal State University-San Bernardino (CSUSB). The other came from the AAC&U Value rubric on Lifelong Learning, specifically, the two elements called Transfer and Reflection. Both rubrics appeared to offer valuable ways of assessing the metacognition evident in a written reflection. Rather than choose one or the other, we combined the two categories (rows) of the AAC&U Value rubric and the three categories (rows) of the CSUSB rubric. We also decided to use four standards of quality (columns) and discussed terminology resulting in: Beginning or N/A, Emerging/Developing, Mastery, and Exceeding Expectations.

photo of a curriculum meeting — Spring 2019 Biology undergraduate curriculum meeting number 1: creating a rubric to assess metacognition using the newly defined and approved SLO 6.

In the second workshop, our goal was to fill in the performance criteria or behavioral anchors that would make up the rubric. After much discussion, we again decided to leave the rubric big and pilot it in our next meeting to determine whether the AAC&U elements or the CSUSB elements would be preferable.

In our third workshop, piloted the rubric by we scoring a packet of student reflections that had come out of the Biology undergraduate research capstone course the previous year. In practice, the faculty found the two elements of the AAC&U rubric easier to apply and more valuable for differentiating between the quality of metacognition in the student responses. Thus, we reduced the final rubric to those two elements.

chart showing a metacognition rubric for biological sciences — This is the rubric to assess metacognition that came out of Biology’s spring curriculum committee meetings with Biggio Center and Academic Assessment.

Developing the Reflection Questions

In the final workshop, our goal was to draft and finalize questions that would be given to sophomores and seniors in the program. These questions would parallel those already being used in the undergraduate research capstone course. These are the questions the committee created:

What has been your favorite learning moment in your major? Please describe it in detail and explain why it was your favorite.
What were the most useful skills you learned in your major and why?
1. Regarding the skills you listed in question 2: how do you know you learned them? Please provide specific examples.
2. How do you plan to apply these skills in future courses or your career?
As a student, what could you have done to learn more? If you could go back in time and give yourself advice, what you say?
Evaluate your capacity to design an experiment and generate hypotheses. Please provide specific examples of aspects of the scientific process you’re most and least confident about.
Reflect on your view of science. How has your participation in your Biological Sciences major changed your view of science, if at all? Please provide specific examples.
Reflecting on your learning journey, what do you value most about your major curriculum (i.e. the courses you took and the order you took them in)?

This question-writing process concluded the initial phase of the Learning Improvement Initiative as it led to the creation of the instrument the department will use to gather baseline data on the metacognition SLO. Moving forward, all students (roughly 75 majors per year) will complete the questionnaire during their capstone course and the curriculum committee will lead assessment using the rubric we created.

The goal is to have every student scored on the rubric every year beginning with baseline data collection in spring 2020 with students who have not experienced the “treatment” conditions, i.e. courses redesigned by faculty to improve metacognition. Over time, we expect that the faculty development workshops around transparent assignment design, reflective writing assignments, and ePortfolio pedagogy will result in graduates who are more metacognitive and data that reflects the learning improvement.

Re-Defining Metacognition: Generating Faculty Engagement

In the third post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Chris Basgier describes the workshops series led by Office of University Writing and the Biggio Center that helped the department redefine metacognition in such a way that they felt like they could understand it, teach it, and assess it. He also unpacks the value of the new definition and points to the work ahead as Biology embraces ePortfolios as part of a pedagogical strategy to increase metacognition in their students.

by Christopher Basgier, Associate Director of University Writing

In the fall semester of 2018, Lindsay Doukopoulos and I had the opportunity to guide faculty from Auburn University’s Department of Biological Sciences (DBS) through a series of workshops devoted to metacognition. These workshops were a direct response to the “metacognition massacre” that had occurred at the August 2018 faculty retreat, as Dr. Robert Boyd recounted in the first blog post in this series.

Photo of Fall 2018 Biology faculty workshop number one: using the TILT Higher Ed transparent assignment design framework to improve metacognition. — Fall 2018 Biology faculty workshop number one: using the TILT Higher Ed transparent assignment design framework to improve metacognition.

Essentially, DBS faculty were uneasy with the definition of metacognition contained in the department’s student learning outcome (SLO), and unsure how to implement metacognitive activities in their courses. Working together, Lindsay and I decided to use these workshops to introduce faculty to the principles of transparent assignment design, offer guidance on integrating reflective writing into courses, and work with them to redefine the metacognition SLO in more familiar terms. 

Transparent Design

We began with transparent assignment design and reflective writing—rather than the SLO—to generate faculty engagement in metacognition. With concrete such strategies for promoting metacognition under their belts, we decided, faculty would be more invested in redefining the SLO and more willing to commit to aligning their courses to that outcome.

Lindsay led the effort to introduce transparent assignment design to DBS workshop participants. According to Mary-Ann Winkelmes with TILT Higher Ed (2016), transparent assignment design invites faculty to clarify how and why students are learning course content in particular ways. Transparently designed assignments include  

The assignment’s purpose, including the skills they will practice and the knowledge they will gain 
The task, including what students will do and the steps they should take to complete the assignment 
Criteria for success, including a checklist or rubric and examples of successful student work

From our perspective, transparently designed assignments can promote metacognition. They make explicit what is often implicit in course assignments, and they help students see how a given assignment fits within the larger context of a course, and even a curriculum. The best designed assignments show students how to draw on what they already know, and help them imagine future implications of their work. 

We gave faculty ample time during the first workshop to consider how they would revise one or more assignments using the transparent assignment design framework, but we also knew that students needed to take an active role in their learning if they were to enhance their metacognitive capabilities. Therefore, I led a second workshop on reflective writing.  

Reflective Writing Component

In Auburn’s Office of University Writing, where I work as Associate Director, we spend a lot of time introducing principles of reflective writing to faculty, namely because we are in charge of the ePortfolio Project, which is Auburn’s Quality Enhancement Project required for accreditation in the SACSCOC.

The ePortfolios we support are polished, integrative, public-facing websites that students can use to showcase their knowledge, skills, and abilities for a range of audiences and purposes. A key component of ePortfolios, reflective writing is a metacognitive practice that invites students to articulate learning experiences, ask questions, draw connections, imagine future implications, and repackage knowledge for different audiences and purposes. After introducing DBS faculty to various levels of reflective writing, I gave them time to develop a reflective writing activity that would support a project or experience already in play in the courses.  

Our hope in these first two workshops was to give DBS faculty practical tools for promoting metacognition in their courses that would not require wholesale course redesign. Transparent assignment design and low-stakes reflective writing are fairly easy to implement in most course contexts.

Redefining the Metacognition Learning Objective

Our third workshop required more intellectual heavy lifting, as it focused on redefining the metacognition SLO. The original metacognition SLO read as follows: 

Students will develop metacognitive skills and be able to distinguish between broad categories of metacognition as applied to their major. In particular, they will distinguish between foundational (i.e., knowledge recall) and higher order (i.e., creative, analysis, synthesis) metacognitive skills. 

The trouble with this definition is that it seems to require students to be able to define different kinds of metacognition (which is difficult enough for faculty), rather than put different kinds of metacognition into practice, regardless of whether or not they can name the metacognitive “categories” they are using.

As an alternative, I turned to research by Gwen Gorzelsky and colleagues, scholars in writing studies who developed a taxonomy of kinds of metacognition. In their framework, the richest form of metacognition is constructive metacognition, which they define as “Reflection across writing tasks and contexts, using writing and rhetorical concepts to explain choices and evaluations and to construct a writerly identity” (2016, p. 226). 

Attracted to the notion that metacognition involves reflection on choices and the construction of identity, Lindsay and I tried our hand at a revised definition: 

Metacognition is defined as the process by which students reflect on and communicate about their role in learning. Reflection and communication may include: 1. students’ choices made in response to the affordances and constraints on learning, and/or 2. students’ evaluations of the success of such choices, particularly across tasks and contexts. Ultimately, these activities should help students develop and articulate identities as scientists. 

Our goal in composing this definition was not to suggest to DBS faculty that it was the right one, only that alternatives were possible. During the final workshop, we asked them to review the original SLO as well as our alternative, and then apply some “critical resistance” to each by reflecting on which terms or ideas made sense, which did not, and what language they might like to include. After much discussion, the group developed a revised SLO: 

Students will develop their metacognitive skills. Metacognition is defined as the process by which students reflect on and communicate about their role in learning. Reflection and communication may include: 1. Awareness of choices made in response to the opportunities (i.e., homework, office hours, review sessions) and constraints (i.e., challenging problems, short time frames) on learning, and/or; 2. Evaluation of the success of such choices, particularly across tasks and contexts. Ultimately, these activities should help students develop and articulate their science knowledge and its value to their professional and lifelong learning goals.  

This definition includes some key changes and additions. It eliminates jargon like “knowledge recall” and “affordances” in favor of more accessible language like “opportunities,” which are further defined in parentheses. Faculty also pushed back on the idea that all students should develop identities as scientists. A great number of students who take DBS courses plan to go into medical fields, so instead, they wanted to put the emphasis on science knowledge, a much more portable focus than science identity. They also added the notion of professional and lifelong learning goals to acknowledge the varied contexts in which their science knowledge might be relevant.  

In the end, our metacognition workshop was a success: the department approved the new definition in December 2018, and many commented on how much clearer and easier to implement and assess it appeared. But our work is not done. Faculty still need to integrate metacognition throughout the curriculum—or at least in courses where it is feasible. The department has agreed that ePortfolios are an effective vehicle for doing so.

ePortfolios to Support Implementation

DBS had joined the ePortfolio Cohort (the group of departments and units committed to implementing ePortfolios) in 2017, and have been working steadily on implementation. Valerie Tisdale, the department’s academic advisor, began the effort to introduce ePortfolios in BIOL 2100, a professional practice course for undergraduate biology majors, in fall 2018. Most recently, in spring 2019, DBS faculty applied for and were awarded with a grant to support an intensive summer workshop to further the integration of ePortfolios in support of metacognition and written communication. My colleague Amy Cicchino and I met with three department members—Lamar Seibenhener, Joanna Diller, and Valerie Tisdale—for four weeks in summer 2019. Utilizing the resources of the ePortfolio Project, the departmental team developed a host of materials for a new, required course that asks students to complete their final ePortfolios during their senior year. 

In the interest of transparent assignment design, they also created an ePortfolio “roadmap” that would help DBS majors understand what an ePortfolio is, why it is important for students in the sciences, and where in the curriculum they might encounter artifacts that could be used as evidence of their knowledge, skills, and abilities. The department approved the new course and completed the roadmap at a retreat in late 2019.

At this point, we are awaiting university-level approval of the new course. In the meantime, we are also planning workshops for DBS faculty on designing meaningful assignments that can be used as ePortfolio artifacts. Taken together, these efforts will help DBS support metacognition through ePortfolios in the years to come. 

References

Gorzelsky, G., Driscoll, D. L., Paszek, J., Jones, E., & Hayes, C. (2016). Cultivating constructive metacognition: a new taxonomy for writing studies. Critical transitions: Writing and the question of transfer, 215.

Winkelmes, M. A., Bernacki, M., Butler, J., Zochowski, M., Golanics, J., & Weavil, K. H. (2016). A teaching intervention that increases underserved college students’ success. Peer Review, 18(1/2), 31-36.

Project Beginnings

In the second post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Katie Boyd describes the activities the year prior to the Metacognition Massacre. These early activities started with the Learning Improvement Initiative that marked the beginning of the collaboration between Biology, Office of Academic Assessment, and the Biggio Center. She also describes how the initial definition of the metacognition learning outcome came about, how the department came to a greater understanding of metacognition, and how that understanding prompted a redefinition of what they believe metacognition is and should be within their context.

by Katie Boyd, Associate Director of the Office of Academic Assessment

Luckily, the work of the Department of Biological Sciences to increase their graduating students’ metacognitive skills did not simultaneously begin and end with the “Metacognition Massacre” of 2018! If we back up just one year, the department was coming off of a strong fall faculty retreat and was ready to turn attention to the thoughtful examination of their curriculum and the knowledge, skills, and abilities expected of all students graduating from their program(s).   

In 2017, each undergraduate degree program in the Department of Biological Sciences (Marine Biology, Microbial, Cellular, & Molecular Biology, and Organismal Biology) had two (2) student learning outcomes and they addressed critical reading, information literacy, and communication skills. Metacognition had only just entered the conversation: it had not been a thoughtful component of the curriculum nor was it a learning outcome for their graduating students. The Department of Biological Sciences needed help. 

Partnering with the Teaching & Learning Center

Enter Auburn University’s Biggio Center for the Enhancement of Teaching and Learning, and Office of Academic Assessment.  That fall semester the two offices joined forces to support programs interested in evidencing learning improvement and jointly issued a request for learning improvement proposals.  

The learning improvement initiative was a way for programs to demonstrate a positive impact by showing how investment in innovative curricular experiences could lead to the improvement of student learning.  The Biggio Center and Office of Academic Assessment wanted to help programs evidence this improvement.  Of note, most departments redesign their curriculum too infrequently or do not have data to inform their curricular redesign, thus delaying their ability to showcase the improved preparedness of their graduates. We anticipated that the joint support of a teaching and learning center AND an assessment office would provide programs with many benefits, such as:  

A streamlined approach to aligning assessment processes with curricular innovation(s) 
The possibility of improving their students’ learning 
Strengthened program reputation 
Faculty satisfaction with process and outcome(s) 
Demonstrated good stewardship of departmental/college resources 
Opportunity for presentation/publication

Biological Sciences submitted a proposal asking for support to define, measure, and improve metacognition amongst their graduates and they were chosen as one of six programs to participate in the inaugural cohort of learning improvement teams. Their specific reasons for choosing this outcome effort for their Learning Improvement Project are outlined below: 

Metacognition was an element in the Action Plan developed by departmental  representatives at the PULSE Institute in June 2016. It was selected for the Action Plan  because it was a neglected element in our curricular planning. 
This SLO was a new one on the department-wide list of SLOs, and of all the SLOs was the one with which faculty were least familiar. Specifically, the program felt they would need the most assistance integrating that into their degree programs. 
A final reason was the hope that working with the Office of Academic Assessment and Biggio Center on improving students’ metacognition would eventually provide a model by which Biological Sciences could plan and implement curricular changes for their other SLOs.

Writing the Learning Outcomes

Thus began the learning improvement project and, throughout the Fall semester, the Office of Academic Assessment facilitated a number of Biological Sciences curriculum committee meetings to re-write all of the department’s student learning outcomes (SLOs). The committee made incremental progress with bi-weekly meetings led by the Department’s intrepid chair.  The department chair quickly led the committee to write six of the seven student learning outcomes, but conversation continued around the metacognition outcome. 

Photo: Bob Boyd showcases Biology's Learning Improvement Project at the year one celebration event hosted by Biggio Center and Academic Assessment in fall 2018. — *Bob Boyd showcases Biology’s Learning Improvement Project at the year one celebration event hosted by Biggio Center and Academic Assessment in fall 2018.*

A number of committee members advocated for the importance of metacognition and reflection and admitted to embedding reflection components into weekly lectures and/or assignments.  On another hand, the former department chair  advocated for a definition that would be easily measurable and liked the idea of students being able to identify the level/type of learning being assessed in specific types of questions on exams or similar instruments (knowledge, comprehension, application).  Bloom’s taxonomy drove a lot of this conversation. 

Eventually, the committee finalized a metacognition SLO (6) and completed their list of seven department-wide SLOs (8 or 9 if you include major-specific outcomes).  At the time, the metacognition SLO was defined by the curriculum committee as: 

Students will develop metacognitive skills and be able to distinguish between  broad categories of metacognition as applied to their major. In particular, they will distinguish between foundational (i.e., knowledge recall) and higher order (i.e., creative, analysis, synthesis) metacognitive skills.

The list of outcomes was shared with all program faculty during a fall faculty meeting and they voted to accept the list as the new set of outcomes.  There were few questions regarding the outcomes during this meeting.  However, I think we can all agree that this is pretty typical when these sorts of items/topics are brought up in faculty meetings.   

Creating the Curriculum Map

A secondary goal of the curriculum committee was to draft a curriculum map aligning the new student learning outcomes with the required courses in each of the three undergraduate curricula.  The first few meetings allowed the committee to finalize the list of classes they wanted to include in the map and a subsequent discussion about how accurate a curriculum map would be when drafted by a subset of the faculty.  The curriculum committee entered the curriculum mapping conversations with some apprehension because the faculty in the room did not represent or teach all of the courses within the curriculum map. 

Eventually, it was decided that they would draft an aspirational curriculum map in which the ideal alignments would be suggested and discussed in a future faculty retreat. When it came to the metacognition outcome, the committee strongly felt as though it should be covered in each required course and that each course truly should be contributing to the students’ lifelong learning.   

Starting to Consider Assessment

With a set of student learning outcomes agreed upon, and a drafted curriculum map, the learning improvement conversation finally began to move towards assessment and measurement. Essentially, there needed to be a way to evaluate whether students were thinking about thinking and knowing about knowing. Enter the Office of University Writing.  It was at this point that Biological Sciences seriously considered ways in which ePortfolios could be used to both teach and assess metacognition.

Initial conversations targeted ePortfolios as a way to encourage reflective writing and simply “house” student assignments. This idea has blossomed and become much more than a data warehouse, and Chris Basgier (Office of University Writing) will expand on this in the next blog post.    This brings us to the Fall 2018 faculty retreat, which allowed for a guided and thoughtful discussion around each outcome and the aspirational curriculum map.  It was this thoughtful discussion that led to the very effective massacre of SLO 6, ultimately pointing to the need for a better definition of metacognition as a learning outcome. 

“The Metacognition Massacre”

In the first post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Bob Boyd reflects on 2018 faculty retreat where Biology faculty rejected responsibility for teaching metacognition in their courses. He also shares where and how Biology’s journey to learning improvement around metacognition began.

By Robert Boyd, Professor of Biological Sciences and former Undergraduate Program Officer for Department of Biological Sciences (DBS). Currently, Associate Dean for Academic Affairs, College of Sciences and Mathematics

My most memorable moment regarding metacognition occurred at a departmental faculty retreat in August 2018, right before the start of Fall Semester. Before this retreat, our departmental Curriculum Committee had created an “aspirational” curriculum map that purported to show which required courses addressed our brand-new list of eight or nine Student Learning Outcomes (SLOs) for each of the three majors in our department.

The Massacre

Metacognition, our new SLO 6, was selected as being a part of every required course. At the retreat, breakout groups were assigned to discuss and describe some aspects of several SLOs (one SLO per group, including a group assigned to “metacognition”) and put their ideas on a flipchart. When all the breakout groups reported, the metacognition group presented a blank flipchart page and said that they had been unable to decide what metacognition was.

Later during the retreat, when we discussed our “aspirational” curriculum map to convert it into a map that showed which SLOs were actually addressed in our core classes, almost all the checkmarks for metacognition were removed from the map. We asked faculty to place Post Its over the SLOs that they didn’t feel like their courses needed to address. In my mind, that retreat was a metacognition massacre. It showed that we needed to do some serious work to define that SLO as well as decide how to integrate it and measure it in our curricula.

Photo of a chart showing a curriculum map from a faculty retreat

Image 1: Biology’s ideal curriculum map presented at the 2018 Retreat. Faculty used slips of pink paper to indicate rejection of an SLO they didn’t think their individual course (left hand column) addressed. SLO 6, metacognition, was almost entirely stricken from the curriculum.

This blog series will present my department’s work on metacognition, mainly focusing on how we have proceeded since the memorable metacognition massacre at that faculty retreat. But I want to take some time now to set the stage by describing my department and some of our work prior to that retreat.

Setting the Stage for the Metacognition Massacre

Auburn University is a land-grant school with about 30,000 students, and has recently achieved the status of a Carnegie R1 institution (meaning that research is an important part of our mission). My department of 43 faculty is a Biological Sciences department, and our courses are vital to the university’s educational mission as well. As evidence of this, in an academic year we teach about 45,000 student credit hours.

This new outcome effort began in January 2016, when one of our faculty, Jason Bond, became Chair and encouraged us to review our curricula, something that had not been done since 2008. Coincidentally, also in January 2016, our department was invited to participate in an NSF-funded Institute at Wofford College in South Carolina designed to help us begin the process of reviewing and revamping our programs.

In June 2016, a small departmental team attended the retreat which was focused on a report by the American Association for Advancement of Science (AAAS) on undergraduate biology education in the US. The report, entitled “Vision and Change in Undergraduate Biology Education: A Call to Action” (referred to as V&C below) and available from this link (https://live-visionandchange.pantheonsite.io/wp-content/uploads/2013/11/aaas-VISchange-web1113.pdf), pointed out that undergraduate biology education needed reform and the workshop involved assessing our department and its curricula.The assessment used a rubric that listed “Student Metacognitive Skills” as one of the ten elements evaluated, with an exemplary department described as “Instructors regularly integrate practice of effective metacognitive strategies within assignments. Most students become adept at reflecting upon, and improving, their own learning and coaching their peers.”

To begin the work of moving as a department from having no outcomes related to metacognition to one that placed it squarely in the SLOs for all of our programs, we held a retreat in 2017 which focused on High Impact Practices (HIPs). This retreat was facilitated by two nationally known educational leaders: Dr. Ellen Goldey (Dean, Wilkes Honors College, Florida Atlantic University) and Dr. April Hill (Chair, Department of Biology, University of Richmond). Faculty engagement at this event was strong and led us to begin the work of formally committing to a curriculum that would address metacognition as an outcome of our undergraduate programs.

In the spring of 2018, we held faculty meetings to introduce V&C concepts and ask the faculty in each of our three majors to evaluate our programs. In every case we decided we were at a “Beginning” stage. According to the V&C rubric, this means “Rarely are students encouraged to reflect on their learning strategies and skills. Study strategies, when discussed, may not be specifically geared to STEM learning or the particular student’s needs.” These meeting led to the 2018 Faculty Retreat described earlier which showed us how challenging it would be for us to understand and embrace our metacognition SLO.

Citations Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science, Washington, DC.

The Evolution of Metacognition in Biological Sciences

By Lindsay Doukopoulos, Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning at Auburn University, and blog mini-series editor.

Much of the literature on metacognition focuses on strategies that faculty can use to improve metacognitive skills in their students and the benefits of such skills. Our mini-series tackles a different kind of problem: how can a department redesign its curriculum to improve metacognition for all students and how will it know if improvement has actually occurred? We believe our efforts can inform others across a variety of disciplines.

Our answer to this question takes the form of a case study in five parts about our collaborative and ongoing efforts to redesign the Department of Biological Sciences’ undergraduate curriculum and program assessment with a goal of improving metacognition for its students and demonstrating that improvement with data. We use a narrative structure to present the key inflection points in this process as well as lessons learned and best practices from our diverse perspectives.

Our collaborators include: Associate Dean for Academic Affairs for the College of Sciences and Mathematics, Bob Boyd (also a Biological Sciences professor and formerly the department’s Undergraduate Program Officer); Associate Director of Academic Assessment, Katie Boyd; Associate Director of the Office of University Writing, Chris Basgier; Chair of the Department of Biological Sciences, Scott Santos; and Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning, Lindsay Doukopoulos.

This timeline provides an overview of our efforts while our individual posts go into more detail about specific strategies and outcomes:

Ideation:

June 2016: Department leaders attend PULSE Institute and decide to make metacognition a student learning outcome (SLO) for all undergraduate programs the Department of Biological Sciences (hereafter, Biology)

May 2017: Program assessment reports at this time include only two student learning outcomes (metacognition not one of them) for each of the three undergraduate programs in Biology

August 2017: Faculty retreat led by NSF Vision & Change experts introducing metacognitive teaching strategies

Commitment:

October 2017: Learning Improvement Initiative launched by Biggio Center and Office of Academic Assessment: Biology proposes to improve SLO 6 – Metacognition

Spring 2018: Biology’s curriculum committee develops a plan for improvement and creates an ideal (“aspirational”) curriculum map to share at the 2018 fall faculty retreat

Lindsay Doukopoulos leading faculty development on metacognition at the 2018 Biology Faculty Retreat

Conflict:

August 2018: Faculty retreat, aka “Metacognition Massacre” – widespread faculty rejection of the metacognition SLO on the curriculum map

A New Approach:

Fall 2018: A three-part workshop series created by Office of University Writing (OUW) and the Biggio Center leads faculty to redefine the metacognition SLO and introduces strategies to support faculty teaching

Turning Point:

December 2018: Outcomes of the workshop series, including the new definition of SLO 6, are presented at a faculty meeting and the faculty vote to approve the new definition

Assessing Metacognition:

January – April 2018: Office of Academic Assessment and the Biggio Center lead Biology’s curriculum committee in creating a metacognitive questionnaire for graduating students and a rubric to assess the level of metacognition evidenced in the responses

Improving Metacognition:

Summer 2019: Biology invests in comprehensive strategy to promote metacognition across the curriculum using ePortfolios and several faculty participate in an intensive course redesign program

What now?

Fall 2019: OUW and Biggio provide ongoing support of teaching interventions to improve metacognition; Office of Academic Assessment provides ongoing support of the assessment of this work

What’s next?

Spring 2020: Gather baseline data on graduates’ metacognitive capabilities Goals: Based on our efforts and an ongoing collection of data, we expect to see increases in students’ metacognitive abilities over time

Taking regular thinking-about-thinking breaks

By John Draeger, SUNY Buffalo State

As the new semester gets underway, I am mindful of the importance of helping students learn critical thinking and metacognition alongside important content. I teach courses in philosophical ethics. My goal is to help students learn to reflect on basic questions of life by critically examining fundamental values. I want students to learn to uncover the underlying substructure of moral issues (Draeger 2014). I want them to see the real-life relevance of fundamental values (e.g., respect, fairness, and individual well-being) and how reflecting on those values inform how they interact with those around them.

We began this semester with Martin Luther King Jr.’s “Letter from Birmingham Jail” because the essay underscores the real-world relevance of philosophy. We move from discussions of King to conversations about #BlackLivesMatter, redlining, and concerns about the criminal justice system. Students are eager to talk about these issues. The trick, however, is to help them learn to think about them in more philosophically nuanced ways. It is tempting to fall into familiar patterns of thinking without taking a hard look at the nature of those thoughts. It is also easy to appeal to authority. King was on the “right” side of history. There is national holiday in the United States dedicated to his memory and a monument on the national mall. But, more deeply, why is racial segregation unjust? Why think we have a moral duty to overturn it? What are the implications of King’s analysis to contemporary debates?

In an attempt to help students learn how to uncover additional layers of philosophical insight, I offer what I call the “philosophical method.” It is a recursive process where students identify the core values, ask critical questions (e.g., about meaning, assumptions), apply to concrete cases, formulate insights gleaned from those applications, and use them to refine their analysis.

At the start of each class, we identity the relevant values under discussion and how they are related to class readings. As we move through a class session, we take frequent “thinking-about-thinking breaks” to put a pause on the discussion to isolate elements of the process. This pause is sometimes even indicated by making a corny “T” gesture with my hands. The idea is that we want to be explicit about the thinking process (i.e. engage in metacognition) in order to understand how the various parts are related and how to engage in deeper reflections about values (critical thinking).

For example, King argues that segregation is unjust because it degrades human dignity, because it harms the human spirit, because policies were arbitrarily applied, and because not all people had a voice in policy creation. It is all too easy to look at this collection of arguments and say “yup, segregation is bad.” However, each of reason points to a different fundamental value. Degrading dignity is related to a duty to respect all people. Harming individuals is related to the importance of personal well-being. Concerns about arbitrary policy application is a worry about fairness. And concerns about voice in a process points to the value of democracy. These values are interrelated, but identifying their separateness is an important step towards exploring their interconnections. And this is precisely what reflecting carefully about basic value questions is all about.

In a previous post, I explored differences between two forms of thinking about thinking, namely critical thinking and metacognition (Draeger 2015). I’ve argued that “critical thinking involves an awareness of mode of thinking within a domain (e.g., question assumptions about gender, determine the appropriateness of a statistical method), while metacognition involves an awareness of the efficacy of particular strategies for completing that task.” The philosophical method that I just described is an example critical thinking within a discipline. However, when I explicitly bring student awareness to their engagement in the critical thinking process we move to a more metacognitive domain. I would argue that these think-about-thinking breaks can be one way of practicing both skills.

Thinking-about-thinking breaks focus on whether our strategies for disciplinary thinking are working. Being metacognitive about our critical thinking processes, for example, can help us see where misunderstandings arise and when we might need to pivot to a new way of approaching the interrelationships between important values. If students have trouble understanding the difference between respect, fairness, and well-being, then metacognitive awareness can suggest a shift in critical thinking strategy (e.g., application to dissimilar case, consider underlying assumptions, explore evidence of their importance, clarify meaning).

Because metacognition is a skill, thinking-about-thinking breaks give students an opportunity to practice in class with both instructor and peer support. These breaks are part of a broad effort on my part to promote metacognitive development alongside content and disciplinary thinking. I encourage metacognitive reading strategies (Draeger 2017) and use Just-in-Time teaching strategies to both encourage metacognition outside of class and inform discussion within class (Draeger 2016). Taking class time to practice metacognitive strategies underscores its importance in the intellectual life of my students. I hope that making the connection between metacognition and critically thinking about fundamental values (e.g., respect, fairness, well-being) will underscore the ways that metacognition is essential to a deeper understanding of the basic questions of life.

References

Draeger, J. (posted July 11, 2014). Using metacognition to uncover the substructure of moral issues.” Retrieved from https://www.improvewithmetacognition.com.

Draeger, J. (posted October 12, 2015). “Two forms of ‘thinking about thinking’: Metacognition and critical thinking.” Retrieved from https://www.improvewithmetacognition.com.

Draeger, J. (posted November 17, 2016) “Promoting metacognitive reading through Just-in-Time Teaching.” Retrieved from https://www.improvewithmetacognition.com/promoting-metacognitive-reading-just-time-teaching/

Draeger, J. (posted June 16, 2017). “Metacognitive Reading Boosts Philosophy Exam Scores.” Retrieved from https://www.improvewithmetacognition.com/developing_student_metacognition_draeger/

How do you know you know what you know?

by Patrick Cunningham, Ph.D., Rose-Hulman Institute of Technology

Metacognition involves monitoring and controlling one’s learning and learning processes, which are vital for skillful learning. In line with this, Tobias and Everson (2009) detail the central role of accurate monitoring in learning effectively and efficiently. Metacognitive monitoring is foundational for metacognitive control through planning for learning, selecting appropriate strategies, and evaluating learning accurately (Tobias & Everson, 2009).

Hierarchy of Metacognitive Control, with Monitoring Knowledge at the bottom, followed by Selecting Strategies, Then Evaluating Learning, with Planning at the top

Figure 1 – Hierarchy of metacognitive regulatory processes. Adapted from Tobias and Everson (2009).

Unfortunately, students can be poor judges of their own learning or fail to engage in the judging of their learning and, therefore, often fail to recognize their need for further engagement with material or take inappropriate actions based on inaccurate judgements of learning (Ehrlinger & Shain, 2014; Winne and Nesbit, 2009). If a student inaccurately assesses their level of understanding, they may erroneously spend time with material that is already well known or they may employ ineffective strategies, such as a rehearsal strategy (e.g., flash cards) to build ROTE memory when they really need to implement an elaborative strategy (e.g., explaining the application of concepts to a new situation) to build richer integration with their current knowledge. This poor judgement extends to students’ perceptions of the effectiveness of their learning processes, as noted in the May 14^th post by Sabrina Badali, Investigating Students’ Beliefs about Effective Study Strategies[. There Badali found that students were more confident in using massed practice over interleaved practice even though they performed worse with massed practice.

Fortunately, we can help our students to develop more accurate self-monitoring skills. The title question is one of my go-to responses to student claims of knowing in the face of poor performance on an assignment or exam. I introduced it in my April 4^th blog post, Where Should I Start with Metacognition? It gently, but directly asks for evidence for knowing. In our work on an NSF grant to develop transferable tools for engaging students in their metacognitive development, my colleagues and I found that students struggle to cite concrete and demonstrable (i.e., objective) evidence for their learning (Cunningham, Matusovich, Hunter, Blackowski, and Bhaduri, 2017). It is important to gently persist. If a student says they “reviewed their notes” or “worked many practice problems,” you can follow up with, “What do you mean by review your notes?” or “Under what conditions were you working the practice problems?” The goal is to learn more about the students’ approach while avoiding making assumptions and helping the student discover any mismatches.

We can also spark monitoring with pedagogies that help students accurately uncover present levels of understanding (Ehrlinger & Shain, 2014). Linda Nilson (2013) provides several good suggestions in her book Creating Self-Regulated Learners. Retrieval practice takes little time and is quite versatile. Over a few minutes a student recalls all that they can about a topic or concept, followed by a short period of review of notes or a section of a book. The whole process can be done individually, or as individual recall followed by pair or group review. Things that are well-known are present with elaborating detail on the list. Less well-known material is present, but in sparse form. Omissions indicate significant gaps in knowledge. The practice is effortful, and students may need encouragement to persist with it.

I have used retrieval practice at the beginning of classes before continuing on with a topic from the previous day. It can also be employed as an end-of-class summary activity. I think the value added is worth the effort. Because of its benefits and compactness, I also encourage students to use retrieval practice as a priming activity before regular homework or study sessions. Using it in class can also lower students’ barriers to using it on their own, because it makes it more familiar and it communicates the value I place on it.

Nilson (2013) also offers “Quick-thinks” and Think Aloud problem -solving. “Quick-thinks” are short lesson breaks and can include “correct the error” in a short piece of work, “compare and contrast”, “reorder the steps”, or other activities. A student can monitor their understanding by comparing to the instructor’s answer or class responses. Think Aloud problem-solving is a pair activity where one student talks through their problem-solving process while the other student listens and provides support, when needed, for example, by prompting the next step or asking a guiding question. Students take turns with the roles. A student’s fluency in solving the problem or providing support indicates deeper learning of the material. If the problem-solving or the support are halting and sparse, then those concepts are less well-known by the student. As my students often study in groups outside of class, I recommend that they have the person struggling with a problem or concept talk through their thinking out loud while the rest of the group provides encouragement and support.

Related to Think Alouds, Chiu and Chi (2014) recommend Explaining to Learn. A fluid explanation with rich descriptions is consistent with deeper understanding. A halting explanation without much detail uncovers a lack of understanding. I have used this in various ways. In one form, I have one half of the class work one problem and the other half work a different problem or a variant of the first. Then I have them form pairs from different groups and explain their solutions to one another. Both students are familiar with the problems, but they have a more detailed experience with one. I also often use this as I help students in class or in my office. I ask them to talk me through their thinking up to the point where they are stuck, and I take the role of the supporter.

The strategies above provide enhancements to student learning in their own right, but they also provide opportunities for metacognitive monitoring – checking their understanding against a standard or seeking objective evidence to gauge their level of understanding. To support these metacognitive outcomes I make sure to explicitly draw students’ attention to the monitoring outcomes when I use pedagogies to support monitoring. I am also transparent about this purpose and encourage students to seek better evidence on their own, so they can truly know what they know.

As you consider adding activities to your course that support accurate self-assessment and monitoring, please see the references for further details. You may also want to check out Dr. Lauren Scharff’s post “Know Cubed” – How do students know if they know what they need to know? In this post Dr. Scharff examines common causes of inaccurate self-assessment and how we might be contributing to it. She also offers strategies we can adopt to support more accurate student self-assessment. Let’s help our student generate credible evidence for knowing the material, so they can make better choices for their learning!

References

Chiu, J. L. & Chi, M. T. H. (2014). Supporting Self-Exlanation in the Classroom. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.). Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php

Cunningham, P., & Matusovich, H. M., & Hunter, D. N., & Blackowski, S. A., & Bhaduri, S. (2017), Beginning to Understand Student Indicators of Metacognition. Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. https://peer.asee.org/27820

Ehrlinger, J. & Shain, E. A. (2014). How Accuracy in Students’ Self Perceptions Relates to Success in Learning. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.). Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http ://teachpsych.org/ebooks/asle2014/index.php

Nilson, L. B. (2013). Creating Self-Regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills. Stylus Publishing: Sterling, VA.

Tobias, S. & Everson, H. (2009). The Importance of Knowing What You Know: A Knowledge Monitoring Framework for Studying Metacognition in Education. In Hacker, D., Dunlosky, J., & Graesser, A. (Eds.) Handbook of Metacognition in Education. New York, NY: Routledge, pp. 107-127.

Winne, P. & Nesbit, J. (2009). Supporting Self-Regulated Learning with Cognitive Tools. In Hacker, D., Dunlosky, J., & Graesser, A. (Eds.) Handbook of Metacognition in Education. New York, NY: Routledge, pp. 259-277.

Awareness of Fractals Strengthens Metacognition Needed for Enacting Informed Teaching Philosophies

by Dr. Ed Nuhfer, California State Universities (retired)

Since 2002, I’ve written a theme-based column, “Developers’ Diary,” for The National Teaching and Learning Forum (NTLF). The central theme through all of these columns is “educating in fractal patterns.” Additionally, I facilitated week-long retreats from 1993 to 2010, and still run workshops, both of which employ visualization of a fractal generator as an aid to understanding concepts of teaching and learning. The wonderful “LAMP” (Learning Actively Mentoring Program) program at the University of Wyoming, where I serve as a mentor, continues to incorporate this aid in participants’ development of informed teaching philosophies.

In writing involved with our academic professions, perhaps no documents are so much the products of metacognition as our written teaching philosophies. These come from within us, which may account for their being so challenging to write. The information-gathering and evidence-based kinds of education through which we mastered most of our own education rarely gave us much practice for metacognitive self-assessment and deep self-reflection.

When properly used, the value of a teaching philosophy lies in “shaping” and nurturing the continuous growth of its author’s expertise. Rather than just a statement, the document serves to direct the author’s intention to enact the practices espoused in the philosophy. In this column, I seek to infuse readers’ already developed metacognitive capacities with an added dimension of “fractal awareness.”

Fractals: Why “Y” Why?

A fractal form is one that develops through growing from a “seed” called a generator (Fig. 1). Development involves repeatedly connecting additional generators to the growing structure. Thus, the character of the full form depends on the characteristics of the generator. A generator consists of simple Euclidean parts, perhaps the simplest being a straight-line segment. We enlist Figure 1 to clarify how initiators form generators, and fractal forms grow through recursively adding more and more generators.

Four levels of fractal development: initiator, generator, fractal form, complex fractal form

Figure 1. Development of a branching fractal form from a “Y-shaped” generator and its precursor initiator (from Nuhfer, 2007). Fractal shapes are the most common of all natural forms. Plants, mountains, clouds, coastlines, patterns of natural events in time like rainfall and floods, blood vessels, and the neural networks in our brains are examples of natural fractal forms.

The concept of fractal form is more than an abstract visualization that inspires creatively thinking about the process of becoming educated. The neural connections that develop in our brains through learning really are fractal forms. When we learn, we connect and stabilize fractal neural networks, so a good deal of our thinking and behavior almost surely has fractal qualities. We can enhance our understanding of educating and becoming educated by discovering the fractal qualities that these endeavors exhibit. One of the most important to recognize is that healthy final forms grow from robust generators. In practice, we can build a sturdy generator from a “blueprint” established by writing a well-informed teaching philosophy. If we mindfully practice this philosophy, the strengths and omissions of our “generator” grow into the strengths and blind spots that characterize our practice.

The branching fractals that develop in our brains are certainly more complex than the model in Figure 1, but even that simple figure helps us to understand and explain countless aspects of the process of learning and, over time, developing higher level thinking capacities.

The Philosophy as a Fractal Generator for Teaching, Learning, and Thinking

The statement, “Metacognition is thinking about thinking” always triggers the question, “What do we think about?” The fractal generator (Fig. 2) in use by me for about the past two decades tends to trigger six items for consideration in what to “think about” to build an informed philosophy. The meaning of “informed philosophy” extends beyond a document informed by a solid base of research on teaching, learning, and thinking. The term “informed philosophy,” as used here, is a document that reflects the growing understanding of ourselves in concert with our growth in knowledge, skills, and evidence-based practices.

Three components in blue (Fig. 2) are mostly components of skills and knowledge. Development of strengths in these three areas comes mainly (not wholly) from external sources. These include the research provided from the literature and from our network of colleagues who help us to build our content expertise and our awareness of varied pedagogical approaches and assessment practices. These originate primarily from resources from outside self, and we mostly develop our practice by drawing on these contributions.

Illustration of components (thinking, teaching, learning) in the fractal generator for faculty and students (by Ed Nuhfer)

Figure 2. A fractal generator model for higher education begins with an initiator that is affect. No deliberate efforts to teach or learn are devoid of affective qualities. Without affective desire to learn to value any of the six areas, such areas will not develop. Practice will then grow from a stunted generator.

The components in red that we call “internal strengths” (Fig. 2) require understanding that develops primarily from within us. The initiator for our generator (Fig. 2) is the red line segment at the base of the generator, which represents our affective feelings. Strong affective interest and enthusiasm may be our most valuable assets for guiding learning efforts to success. We needed to want to do something such as attend college, major in an area that felt attractive and to continue acting to achieve expertise by persevering to develop. That desire comes from within. When our affective passion and cognitive focus align for learning, we are unlikely to fail.

Finding Our Initiators from Within

In starting to write a teaching philosophy, a valuable awareness occurs when we query ourselves about how we obtained our present affective desires for what we aspire to do. Recalling an influential mentor often reveals from whom, when, and where that initial desire occurred. Recollecting a mentor’s valued qualities often reveals that how a teacher now hopes to be remembered began to form with learning to appreciate the power and validity of a particular mentor’s qualities. These recollections usually carry strong emotional ties, and early ideas that produced our conceptions of what constitutes good teaching can be beneficial if they really fit us. They can also be limiting if we unconsciously attempt to imitate a revered mentor rather than advance to develop the teaching that arises from our unique experiences and values.

Cultivating the habit of regular metacognitive conversations with ourselves allows us to confront a query of great importance: “Is what I am doing in the present truly what I most intended to do?” If not, the revised philosophy serves to direct our efforts back to regain doing what we intended to do. That practice allows us to tap the optimal power of affect by doing what a plan of deep introspection revealed that we most wanted to do in our practice. When a troubling event starts to occur, a valuable first reflection is, “Am I actually practicing my philosophy through how I am engaging with this challenge?” Often, we will find that troublesome events occur from a brief moment of inattention that sidetracks us into doing something other than what we intended to do.

Fractals and Uniqueness

In the neural networks that store the well-developed expertise within our brains, the separate neural components are in communication with one another, and they enlist one another to engage successfully with challenges or unexpected changes. Thus, the six areas of the generator (Fig. 2) that grow through our experience should grow to work simultaneously in active practice. Although I’ve found no contributions to research in faculty development that cannot be addressed from within the components of Figure 2, the fractal model is not one of prescriptive development. It does not lead to producing instructors in cookie-cutter fashion who all think alike and teach alike. Indeed, it cannot.

For the same reason that there are neither two trees nor two rainstorms that are alike, there can be no two brains that wire alike. Small differences between individuals’ generators occur through the unique experiences of each person. As these differences influence the replication through the repeated exercise of one’s practice, they guarantee the development of diversity and uniqueness of every teacher, every student, and thus every teaching moment experienced within a class. An internalized awareness that these will never occur again leads to consciously respecting others and valuing the present moment deeply.

We have seen in this brief entry how becoming aware of the pervasiveness of fractals in the physical world and understanding the role of the generator helps the author appreciate the utility of a written teaching philosophy for illuminating one’s own generator. Through the recursive process of repeated implementation, robust generators significantly strengthen one’s practice through time. In our next blog entry, we will examine metacognition’s specific roles in developing each of the six individual components.

Nuhfer, E. B. (2007). “The ABCs of fractal thinking in higher education.” To Improve the Academy (25) 70-89.

Creating a Proactive Transition for the College Student with LD (Part lll): An Elevator Pitch and the Two O’s

By Mary L. Hebert, PhD; Campus Director, The Regional Center for Learning Disabilities; Fairleigh Dickinson University

I have submitted earlier posts (Part 1; Part 2) that have addressed the transition for high school seniors with a learning disability (LD). I’d like to further propose two concepts from the counselor corner of my work with students with learning disabilities and executive function challenges as they navigate their new college learning environment: an elevator pitch and the two O’s.

Elevator Pitch spelled out in colored blocks

Points of transition, whether perceived as positive or negative, are typically experienced as stressors just by design of being human. Transitions are potentially more stressful for students who have spent a learning career managing an LD. Anticipatory responses to a transition can include anxiety and concerns about navigating the pace and content of a new academic environment. For a student with an LD, this can feel not just like a change of pace, but rather a frenzied experience without proper preparation.

Metacognition offers an outstanding framework for preparing for this new learning environment. Self-reflection and intrapersonal awareness as far as how the LD has impacted one socially, cognitively and emotionally is an excellent endeavor in order to prepare for the requisite independence of mind and action to tackle the adjustment ahead in college.

Students who have had a documented LD during their k-12 years experience concerns developmentally typical of all new college students:

Will I succeed in this new environment?
Will I make new friends?
How will I manage on my own?

Students with LD, however, sometimes may experience more significant concerns as a result of their prior educational experiences. As these high school seniors transition, they will need to prepare for a new learning environment, one where they are starting everything anew and independently. They will not have the familiar support and structure of a case manager, parents, clearly demarcated schedule encompassing their entire day, or other familiar assistive supports that helped them navigate the terrain of their high school educational experience.

In this post I will focus on two concepts that I have utilized during my time as a counselor for college students with LD. Both of these are transition “tools of mind” that provide a metacognitive orientation to adjustment to college life. The first is the importance of having an Elevator Pitch at the ready upon entry to college. The second is the awareness and reflection on The Two O’s: opportunities and obstacles. As stated in my prior posts, my mission is to support students by helping them prepare, which will ease transition stress and increase readiness. Preparation prevents perspiration!

The Elevator Pitch

We have all heard this expression as it relates to the opportunities in business and ‘selling oneself’ for a position when one does not have much time to pitch their fit for a job. In the case of a student with LD, they will need to be able to independently articulate their needs to relevant others in the college setting. For students with LD it may be challenging to speak in an impromptu fashion with individuals they do not know well. A prepared elevator pitch will help them in such situations.

The elevator pitch becomes particularly important when a student will need to advocate on their own behalf. Self – advocacy skills are significantly associated with success in the college setting. Having a parsimonious, prepared statement of one’s needs at the ready can be advantageous for the student with a LD entering a new learning environment and adjusting to more independent self-advocacy.

An accurate self- assessment or metacognitive reflection of one’s strengths, skills sets and challenges is essential for academic as well as future career selection. Often times, students who have moved through their education with an LD have had to focus significantly on tackling skills sets such as reading, writing, math and other core academic skills. This focus can take away from time spent considering their goals and strengths, which should be the foundation for self-advocacy. Solid self-advocacy improves the likelihood for a gratifying personal and career experience (Palmer and Roessler, 2000).

I suggest that students be proactive and prepare a metacognitive reflection of their LD, characteristics of its impact on their academics, and what they know to be helpful in their educational environment. It is also key for them to become knowledgeable about college-level accommodations and the rights they will have in college to seek out services for their learning needs. It is advantageous to apply metacognition in a way that will foster an opportunity to reflect and prepare a succinct, effective pitch that achieves key goals as they adjust to their new learning environment. These key goals include:

Self Advocacy
Self Awareness
Self Efficacy

I like to think of these three goals as the ultimate selfies! The ability to convey their learning needs and goals to their disability coordinator, a professor, a tutor or another professional in their college environment will be essential to have at the ready. Doing so will decrease stress and increase the ultimate selfies.

Obstacles and Opportunities (the two O’s)

There will be both opportunities and obstacles. Simply and plainly, there is no escaping either for ANY student. Preparing in a metacognitive manner about both these types of eventual experiences will benefit any student but particularly a student with a history of LD. Provide metacognitive reflection prompts by asking these or similar questions of your student:

What have been some successes in your educational career thus far?
What have you learned from these? How have they helped you move ahead in regard to the ultimate selfies?
What have been some obstacles in your educational career thus far?
What have you gained from these challenges? How have they advanced your movement toward your educational goals?

This metacognitive reflection provides the bedrock for continued reflection at the college level. From the counselor’s chair it is a continued dialogue of self-discovery as the student ultimately encounters and reflects on opportunities and obstacles. The reflection prompts also provide a vocabulary to frame experiences that feel elusive (the opportunities) as well as the stressors (the obstacles), and these prompts promote turning the latter into openings for growth. And yes, they contribute to the ultimate selfies.

In conclusion, my wish is that the summer brings forth much needed time for students to relax, and have fun. But, importantly, the summer is also the ideal time to reflect on the path traveled thus far and prepare for the future. Metacognition offers an effective tool to apply to past educational endeavors, pave the way for the next educational transition, and create a foundation for success.

Palmer, C. and Richard T. Roessler (2000). Requesting Classroom Accommodations: Self Advocacy and Conflict Resolution Training for College Students with Disabilities. Journal of Rehabilitation. 66 (3): 38-43

Changing Campus Culture with the Ace-Your-Course Challenge

In the final post of the guest blog series on “Working with Faculty to Promote Metacognition,” Dr. Eric Kaldor discusses lessons learned from the implementation of a campus-wide metacognition program inspired by Saundra McGuire’s work. The associated research project was awarded the Robert J. Menges Award for Outstanding Research in Educational Development by the Professional and Organizational Development (POD) Network.

by Eric Kaldor, Ph.D.; Associate Director, Sheridan Center for Teaching & Learning, Brown University

For many faculty members, the “fact” that some students are just not capable of college-level learning remains part of the taken-for-granted assumptions embedded in the culture of disciplines and campuses. Despite significant efforts to share insights from the scholarship on metacognition and growth mindsets (e.g. Doyle & Zakrajsek, 2013; Dweck, 2016; McGuire, 2015; Nilson, 2013), campus cultures are slow to change, and fixed mindsets continue to dominate many institutions. This post describes efforts to change the culture at the University of Rhode Island, the communication strategy we used, and some lessons learned.

With approximately 14,000 undergraduate students and 1,000 full and part-time faculty, the University of Rhode Island is a challenging setting to advance culture change. Our story began with a conversation with Melvin Wade, former Director of the Multicultural Student Services Center (MSSC). I was working in the Office for the Advancement of Teaching & Learning (ATL) and planning for Saundra McGuire to visit our campus. I was particularly concerned to fill our 1,000-person auditorium with students for her “Metacognition is Key” workshop. When I asked for his advice, Melvin insisted we must ensure her visit had a lasting impact on our campus. Toward this end, we assembled a group of professional staff and graduate students from ATL, the MSSC, the Academic Enhancement Center, First-Year Programs, and Professional Advising. Over a series of conversations, this informal group conceived of something we came to call the Ace-Your-Course (AYC) Challenge. We assumed we would only run the AYC Challenge once as a companion to Dr. McGuire’s workshop. Instead, a snowstorm gave the Challenge a much longer life.

Building on the McGuire Model

We designed the AYC Challenge to extend students’ metacognitive experience and reflections beyond Dr. McGuire’s workshop. We developed the AYC Challenge as four weekly self-assessment surveys (for detailed description see Kaldor & Swanson, 2019) to create additional metacognitive experiences (Flavell, 1979) by encouraging students to:

Test learning strategies relevant to them individually.
Engage in key practices for metacognitive reflection: observation, description, evaluation, and action planning.
Feel part of a larger community working to grow as learners.

When a snowstorm postponed Dr. McGuire’s visit to the next semester, our multi-unit team led her workshop twice using slides and talking points from her book (McGuire, 2015) and invited students to participate in the AYC Challenge. Of the 240 students attending a workshop, 50 completed all four weeks of the challenge. After we shared the positive results from our pilot with faculty members, many encouraged their students to attend Dr. McGuire’s rescheduled workshop in September 2017. Some went further and agreed to share grade data as part of an IRB-approved study to examine how participation affected grades. We specifically identified a set of gateway science courses from Chemistry, Biology, and Nutrition and Food Sciences that have large enrollments of first-year students.

Over 1,000 students attended Dr. McGuire’s workshop with some in remote viewing locations, and 202 of those completed the second AYC Challenge. The self-reported results for this larger group were strikingly similar to those from students in the pilot AYC Challenge when we led the workshops. Holly Swanson and I analyzed final grades for 979 students in the eight gateway science sections (347 attended the workshop and of those 55 completed the challenge) using OLS regression with controls for several predictors of academic performance including high school GPA and exam 1 z-score. Compared with their peers who did not attend the workshop or participate in the challenge, attending the workshop and completing the AYC Challenge was associated with a final course grade half of a letter grade higher (Kaldor & Swanson, 2018).

Inclusive and Extensive Communication

Much of our success originated from a spiral of communication that grew outwards from a core group of professional staff and graduate students who became involved in planning for Dr. McGuire’s originally scheduled visit. Our colleagues working in various student support services helped develop a plan to reach students and motivate them to attend the workshop and participate in the challenge. These colleagues advised us on when to hold the workshop, how to market our efforts, and what kinds of messages would appeal to students.

One critical piece of advice was that students were more likely to attend if instructors offered extra credit. In the faculty development office, we knew that instructors of large enrollment courses would only offer extra credit if it did not add significant work. Using google forms, a mail merge add-on, card swipe readers, and course rosters, we developed a system for students to pre-register, receive reminder emails, and swipe their id cards after the workshop. With this system in place, instructors for over 30 courses received a list of student attendees within a week of the workshop.

To nudge students who attended the workshop to start the Challenge and complete all four weeks, we used two techniques. First, students were told that completing all four weeks would make them eligible for a drawing for ten $100 gift cards to the campus book store. Second, we started the Challenge at the end of the workshop with students selecting one or more strategies to try on a Google form at the end of the workshop.

photo of Ace Your Course Challenge winners — Four of the ten winners of a raffle for students who completed the Ace Your Course Challenge.

The next spiral outwards involved engaging more faculty in a conversation on the powerful ways they could help their students learn. Prior initiatives that had promoted Dweck’s (2016) insights on growth mindsets had primed many faculty and staff for these conversations. Specifically, they wanted to know what else they could do beyond promoting a growth mindset, and a metacognitive approach to learning strategies offered them concrete answers.

In addition, faculty members who had moved away from fixed mindsets about who could succeed in their courses shared their insights on how to approach their still skeptical peers. We developed a strategy of presenting quantitative data alongside student voices to describe the student experience (examples are available here: https://web.uri.edu/atl/ace-your-course-challenge/). Initially, our quantitative data was limited to student self-reports. With the benefit of a snowstorm, we had the chance to organize an IRB approved research project to answer important questions that skeptics raised.

As we shared this data on campus, we were asked to try different permutations of the Metacognition Workshop plus AYC Challenge in two different settings – a support program for conditionally readmitted students and two gateway chemistry courses. In addition, we were asked to offer workshops for professional staff and faculty so they could include McGuire’s approach in their programs and courses.

One of the most successful workshops, “Teach Your Students How to Learn in 50-minutes” provided an annotated version of Dr. McGuire’s slides with breakout discussions about the key messages to motivate students. This led many instructors to experiment with including different elements of her metacognitive approach to learning strategies into their courses.

Some Lessons Learned and Suspected

Each AYC Challenge has generated new data and insights into the potential for URI students to make significant gains in their metacognition. This new data has generated new conversations, which have led to variations on the McGuire workshop and/or the AYC Challenge. This has been a fruitful if unintended process.

Our skeptical internal voice continues to ask how we could nudge more students into participating. We noticed lower participation rates for students from historically marginalized groups in our gateway science course study. This led us to experiment with embedding the workshop plus challenge into courses, but our early experience raised many concerns around overloading instructors and maintaining fidelity with the core AYC challenge experience.

In a promising next iteration, my URI colleague Michelle Fontes-Barros has suggested a partnership with student organizations and clubs, particularly STEM affinity groups for students from historically marginalized groups. Convinced of the value, a student group might sponsor a workshop in a regular meeting space. Student leaders might promote peer commitments to complete the AYC Challenge. Past AYC Challenge participants might help present the workshop and send messages during the Challenge to encourage persistence. This next iteration has the potential to be much more student-centered, but it will be important to critically evaluate the student experience and share results with the wider university community to energize the campus conversation on metacognitive development.

Doyle, T., & Zakrajsek, T. (2013). The New science of learning: How to learn in harmony with your brain. Sterling, VA: Stylus Publishing.

Dweck, C. S. (2016). Mindset: The new psychology of success (Updated Edition). New York, NY: Ballantine Books.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34(10), 906–911.

Kaldor, E., & Swanson, H. (2018, November). A campus-wide strategy to develop metacognition in gateway science courses. Paper presented at the POD Network Conference, Portland, Oregon.

Kaldor, E., & Swanson, H. (2019). How can you elevate metacognition on your campus? Try the Ace-Your-Course Challenge. The National Teaching & Learning Forum, 28(2), 5–7.

McGuire, S. Y. (2015). Teach students how to learn: Strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Sterling, VA: Stylus Publishing.

Nilson, L. B. (2013). Creating self-regulated learners: Strategies to strengthen students’ self-awareness and learning skills. Sterling, VA: Stylus Publishing.

Using Communities of Practice to Support Online Educators in Fostering Student Metacognition in Virtual Classrooms

The second post in the “Working with Faculty to Promote Metacognition” guest series is from educational consultant Valencia Gabay, who writes about establishing communities of practice with faculty at a fully online institution to promote metacognition through the instructors’ own reflections on teaching.

by Valencia Gabay
Educational Consultant, Orlando, Florida
Doctoral Student, Organizational Leadership
Indiana Wesleyan University

In our society, the tides of change force students to be highly motivated, self-directed learners. However, authors Cameron and Quinn (2015) stated, “The implication in education is that we are currently preparing students for jobs that don’t yet exist, to use technologies that have not yet been invented in order to solve problems we don’t even know are problems yet” (p. 9). So, how do we prepare students to flourish in this brave new world? One answer: we inspire them to be intellectually curious and use their metacognitive knowledge. But, we must first tap into our own metacognitive knowledge to support students in doing the same.

picture showing black and orange question marks on a black table

Metacognition is thinking about how you think and the ability to evaluate one’s use of knowledge in learning and decision-making processes (Halpern, 2015). And, like critical thinking, metacognitive skills can be taught even in a virtual learning environment. Our book, Group Coaching and Mentoring: A Framework for Fostering Organizational Change (Algozzini, Gabay, Voyles, Bessolo, & Batchelor, 2017) presented a unique professional development model in which the community of practice approach helped online instructors integrate metacognitive strategies into their instructional practices.

I use the study described in this book to illustrate how instructors, in collective learning spaces, can generate intellectual curiosity and metacognitive energy that is transferable to the virtual classroom (Algozzini et al., 2017). In this study, a faculty director at a fully online university placed 43 online instructors into six communities of practice, each facilitated by a mentor lead. Using web-based conferencing tools, communities of practice met weekly over approximately nine months to discuss information on metacognition and its value to their professional development. Communities of practice cultivated metacognitive energy in two distinct ways.

Using Self-reflection

First, leads used communities of practice to create moments for self-reflection. Instructors assessed their current teaching styles with their peers and examined ways metacognition could influence job performance. Reflection is paramount to enhancing metacognition, but it is essential to know how to question to prompt reflection. According to organizational psychologist and researcher Dr. Tasha Eurich (2017),

Why questions can draw us to our limitations;
What questions help us see our potential.
Why questions stir up negative emotions;
What questions keep us curious.
Why questions trap us in our past;
What questions help us create a better future. (Eurich, 2017, para. 13)

As such, our community leads used the following lines of questioning to encourage instructors to foster a stronger connection with metacognition.

In your own words, how would you define metacognition?
What does metacognition mean for you as an instructor?
What information from the resources about metacognition resonated with you the most?
How can you apply that information in the classroom setting to promote student success?

The reflective questioning sparked a renewed interest in intellectual wellbeing. Instructors saw themselves as learners, became aware of their strengths and weaknesses, and set attainable goals towards self-improvement.

Creating Metacognitively-based Conversations

Second, instructors learned how to fashion metacognitively-based conversations. Those who question critically tend to be strong critical thinkers, and critical thinkers rely on metacognition to ensure their thinking processes will reach desired learning outcomes (Halpern, 2014). Therefore, the community leads challenged instructors to use open-ended questions to keep discussions robust when engaging with their colleagues in communities of practice meetings. Additionally, instructors participated in exercises using Bloom’s Taxonomy to develop question stems that produced higher order thinking.

Impact for Online Instructors

In a survey, instructors reported that communities of practice provided a safe place for learning how to question and evaluate one’s skills. After working in communities of practice, instructors became more confident using metacognition to bridge gaps in their work performance (Algozzini et al., 2017). Most importantly, instructors possessed a model for generating intellectual curiosity and metacognition among their students. It started with teaching them the power of reflective questioning. This change in teaching style emerged through the prism of social, teaching, and cognitive presence (Garrison, Anderson, & Archer, 2005).

Instructors increased their social presence and transformed their virtual classrooms into a community where students felt comfortable reflecting on what they learned. (Here is a collection of Tips for Creating Social Presence in Online Classrooms.) Instructors knew that students possess a teaching presence; they learn with and from each other. Therefore, instructors made learning content more relatable. They incorporated popular global or national issues relevant to the class discussion, so students could apply what they learned to real life examples. Finally, instructors strengthened their cognitive presence by using class forums to host metacognitively-based conversations. They challenged student thinking by asking open-ended questions and pushing them to support their claims with facts (Algozzini et al., 2017). As instructors demonstrated these tactics, students did the same among their peers.

Inspiring students to be intellectually curious begins with us recognizing that we as instructors are also learners who need to invest in our intellectual wellbeing to better serve the population we teach. We want students to know how to reflect and question as they develop into the thought leaders and global thinkers of tomorrow. So, as you prepare your students for this brave new world, consider the following questions: How are you staying intellectually curious? In what ways are you using your metacognitive knowledge to support students to think and to question?

References

Algozzini, L., Gabay,V., Voyles, S., Bessolo, K., & Batchelor, G. (2017). Group coaching and mentoring: A framework for fostering organizational change. Campbell, CA: FastPencil, Inc.

Cameron, K. S & Quinn, R.E. (2011). Diagnosing and changing organizational culture. San Francisco, CA: John Wiley & Son, INC.

Eurich, T. (2017). The right way to be introspective: Yes, there’s a wrong way. Retrieved from https://ideas.ted.com/the-right-way-to-be-introspective-yes-theres-a-wrong-way/

Garrison, D. R., Anderson T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. Internet and Higher Education, 13(1), 1–2.

Halpern, D. F. (2014). Thought and knowledge, An introduction to critical thinking, (5th ed.). New York, NY: Psychology Press.

Who is Qualified to Teach Metacognition?

In the second post of the “Working with Faculty to Promote Metacognition” guest series, Dr. Nirmal Trivedi discusses several ways he helps a diverse set of instructors with varying metacognition experience integrate the topic into their first-year seminar courses. For his work with first-year seminars, Dr. Trivedi received the 2018 Excellence in Teaching First-Year Seminars Award from the National Resource Center for The First-Year Experience and Students in Transition.

by Nirmal Trivedi, Ph.D.
Director, First-Year Seminars and Assistant Professor of English
Department of First-Year and Transition Studies
Kennesaw State University

Who is qualified to teach metacognition? If we agree that teaching the concept often results in improved academic performance, shouldn’t all faculty members be trained on how, when, and why metacognition should be embedded into their courses, regardless of content area?

At Kennesaw State University, we’ve had a unique opportunity to redesign our First-Year Seminar course to include a heavy focus on metacognition. This 3-credit academic seminar, which is largely uniform in content and required of most first-time students, serves approximately 3,500 students each fall semester with anywhere between 65-80 part-time and full-time faculty teaching the course. The vast majority of these instructors do not have a background in psychology of human learning, and many have either never taught college students, or have only taught them beyond the first year.

Thankfully, student testimonials reflecting positive experiences with our seminar’s focus on metacognition have served to intrigue those who are new to metacognition and convinced skeptical faculty of the value of teaching the concept and its practice. Recent popularization of the concept by Professor Emerita of Chemical Education, Dr. Sandra McGuire, has helped to demystify the term for students and educators alike. Her two books, Teach Students How to Learn and Teach Yourself How to Learn, written for faculty and students, respectively, provide guidance for the uninitiated. McGuire effectively shows why metacognition is essential for all educators to know and teach. An answer to how one builds a local cohort of metacognition experts without the disciplinary expertise in educational psychology, however, can be elusive.

As someone new to teaching metacognition—a kind of “metacognition convert” myself—I can relate to the need of faculty for a clear rationale for changing teaching methods. In this post, I outline five steps that we have used at KSU to develop faculty to incorporate metacognition into their own teaching. In our First-Year Seminar, we train faculty on how and when to take each of the steps through an initial training session at the beginning of the semester and by providing template assignments with embedded reflection questions that call for metacognitive thinking. This approach has helped us build a growing cohort of local metacognition advocates.

Step 1: Make Student Learning Transparent to Faculty

Most faculty agree that they want to see more engagement in class material from their students. Most want to see their students read carefully, practice their writing skills, and to self-evaluate how and why certain learning strategies work or don’t work. Perhaps most of all, faculty want more time to discuss ideas and less time guessing how much or how little students have learned by the end of each class session.

In our program, we train faculty to make learning transparent by asking students to write two “takeaways” at the end of a class session. The questions are “what are two points you will remember from today’s lecture and what is one question that you still have about the topic?” The exercise demonstrates what students actually remember from a class session. In our faculty training, we incorporate the “takeaways” after each component of the training to showcase how the exercise works and to help the training presenters to clarify their own message. In seeing how the simple metacognitive exercise can assist faculty in their own learning by making learning transparent, faculty begin to ask about how much of what they themselves teach is actually absorbed by their students.

The takeaways also serve as a useful beginning of the next session’s lecture or discussion. Faculty have found that their own class preparation is significantly simplified since they are better informed as to what their students understand and don’t understand. It’s important that these takeaways remain anonymous to allow for authentic student responses.

Step 2: Relate to the Student Experience

We train faculty to respond to the students’ desire to learn deeply by focusing learning outcomes not only on teaching content, but on how to learn the content. For example, we prepare faculty to show their students how to read the course textbook through strategic skimming, annotating, and self-testing. In one component of the training, each faculty member is taught a short lesson about how to read actively (as if the faculty were the students) followed by a series of student testimonials reflecting on the lesson’s impact.

From the student perspective, it matters less that the faculty member is a metacognition expert than if he or she truly cares about their learning. We know how underprepared many students are for college academics, but we often neglect to understand that many students crave to learn and just need appropriate challenge and support. As one student says in a takeaway after an active reading lesson, “why don’t they teach

Student quote: "Why don't they teach this earlier in the school system?"

Step 3: Conduct a Learning Demonstration

Faculty can be students too. In addition to demonstrating how to teach active reading, we introduce metacognition to faculty through common learning experiments such as the “Count the Vowels” Activity,” or a “Levels of Processing” activity. The goal of these exercises is to emphasize how memory is tied to our brain chemistry.

Faculty tend to value the impact of these exercises as they themselves are in the process of mastering the new content.

Step 4: Demonstrate Value of Process Alongside Content

Of course, it is essential that faculty understand that conceptual knowledge about metacognition is distinct from practicing metacognitive techniques (Pintrich, 2002). The distinction makes way for a productive discussion about the amount of content required for each course.

For some time, teaching scholars have been urging faculty to consider how much specific course content is necessary. The Association of American Colleges & Universities, for example, conceptualized their “Essential Learning Outcomes” (ELO) as balancing content knowledge with skill-based actions. With a course like a First-Year Seminar that makes metacognition and its practice very explicit, achieving an ELO like “developing skills for lifelong learning” quite feasible.

Each of our template assignments and rubrics include prompts for students to explain how they learned the material as well as what they have learned.

Step 5: Make the Case for Equity in Learning Skills

Ultimately, teaching students how we learn can bridge the gap between those who have had opportunities to explicitly practice these metacognition techniques in secondary school and those who have never encountered them before. The responsibility of educators is all the more important since even non-experts in learning theory can learn and disseminate the techniques with minimal training. Elaborative interrogation, self-explanation, and practice testing, for example, are among the most impactful learning strategies and least complicated to teach (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013).

In sum, faculty may at first feel like they have to learn a new field to teach metacognition. We tell them that they may already be teaching students these skills, and that once they see the scholarly basis for these techniques, they can teach them with confidence. In a post-semester reflection on the impact of incorporating metacognition and its practice for the first time with an assignment, one faculty member made a comment that was echoed by several instructors—that “it made more of a difference to my students than any other assignment I’ve ever taught.”

References

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving Students’ Learning with Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest: A Journal of the American Psychological Society, 14(1), 4–58. https://doi.org/10.1177/1529100612453266

Pintrich, P. R. (2002). The Role of Metacognitive Knowledge in Learning, Teaching, and Assessing. Theory Into Practice, 41(4), 219–225. https://doi.org/10.1207/s15430421tip4104_3

Promoting Metacognition Across the Institution through our Partnerships with Faculty: The Educational Developer’s Role

by Hillary H. Steiner, Ph.D., Kennesaw State University

Dr. Hillary Steiner is the Interim Associate Director for the Scholarship of Teaching and Learning (SoTL) for the Center for Excellence in Teaching and Learning at Kennesaw State University in Georgia, USA. She is our Summer 2019 guest editor for a blog post series that shares case studies across three institutions. These case studies demonstrate that educational developers can be agents of change within their institutions with respect to supporting the development of metacognition.

How can we ensure students know about metacognition? By promoting it to the faculty who teach them. My students often joke that “metacognition” is my favorite word because they hear it so often in the classroom. The faculty on my campus might be starting to think the same thing, as I integrate the concept into so much of what I do. In my dual role as an educational developer and a faculty member with research interests in the application of educational psychology to higher education, I consider myself a metacognition advocate.

My advocacy for metacognition branches out through all levels of my institution—featuring prominently in the activities of our Center for Excellence in Teaching and Learning and trickling into my own classroom through major assignments as well as everyday conversations with students. It is central to my own Scholarship of Teaching and Learning, as well as the SoTL of many of the faculty members with whom I work. In order for metacognition to take hold in an institution’s culture, we must ensure students and faculty know about its power.

Group picture at new faculty orientation at Kennesaw State University

Our university is one of the many who have invited metacognition advocate Dr. Saundra McGuire (McGuire, 2015; 2018) to speak to students and faculty, and the buzz that was created around her visit has generated considerable interest in the concept. Many other colleges and universities have experienced a similar ground swelling of support for the idea’s application to the classroom. Those of us who work in educational development roles can capitalize on this current attention to metacognition by helping faculty who are unfamiliar with the concept realize the importance of a reflective, goal-directed approach to one’s own learning and performance. Ultimately, this advocacy can change the culture of an institution by transforming, in small ways, the way instructors teach and students learn. In this guest blog series on “Working with Faculty to Promote Metacognition,” three authors offer their thoughts on promoting metacognition at the student, faculty, and institutional level through their partnerships with instructors.

First, Dr. Nirmal Trivedi, Director of First-Year Seminars at Kennesaw State University (KSU), writes about the ways in which he helps faculty—many of whom are part-time instructors from outside academia and initially unfamiliar with metacognition—infuse metacognitive practices into their courses, with a goal of changing students’ approaches to studying. First-year seminars at KSU have been transformed to include metacognition as a key focus, which has helped many students successfully navigate the college transition. This transformation earned the program the 2018-19 Momentum Year Award from the University System of Georgia, given to the program that best encourages student achievement in the first year of college.

Second, Valencia Gabay, educational consultant and doctoral student at Indiana Wesleyan University, writes about establishing communities of practice with faculty at a fully online institution to promote metacognition through the instructors’ own reflections on teaching. By focusing on ways the instructors themselves can be metacognitive and using a model from organizational development (Algozzini, Gabay, Voyles, Bessolo, & Batchelor, 2017), she modeled reflective practice in a way that is transferable from instructor to student.

Finally, Dr. Eric Kaldor, Associate Director for the Sheridan Center for Teaching and Learning at Brown University and formerly at the University of Rhode Island (URI), describes the way in which URI took advantage of the buzz surrounding Dr. McGuire’s visit to his campus, creating an institution-wide program that changed the culture of the university at large. Particularly important to this effort was the support and communication provided by various campus partners that made it easier for faculty to understand and implement changes in their curricula.

Readers of this blog series will find useful suggestions to help them ensure that the word gets out about metacognition on their campuses. Educational developers can be agents of change within our institutions because of our relationships with many of the institution’s stakeholders. Through our partnerships with faculty, we have an indirect, but still palpable, influence on student learning (Condon, Iverson, Manduca, Rutz, & Willett, 2016). And as metacognitive practitioners ourselves, we can practice what we preach, engaging in reflective and purposeful analysis of our own messages to the academy about how people learn.

References

Algozzini, L., Gabay,V., Voyles, S., Bessolo, K., & Batchelor, G. (2017). Group coaching and mentoring: A framework for fostering organizational change. Campbell, CA: FastPencil, Inc.

Condon, W., Iverson, E. R., Manduca, C. A., Rutz, C., & Willett, G. (2016). Faculty development and student learning: Assessing the connections. Bloomington, IN: Indiana University Press.

McGuire, S.Y. (2015). Teach students how to learn: Strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Sterling, VA: Stylus.

McGuire, S.Y. (2018). Teach yourself how to learn: Strategies you can use to ace any course at any level. Sterling, VA: Stylus.

Wrapping up Metacognition: Pre- and Post-Exam Interventions

By Jennifer A. McCabe, Ph.D., Goucher College

Multiple studies have demonstrated that college students report using less-than-optimal learning strategies when preparing for exams. Without explicit instruction on effective techniques, along with guidance on how to engage in metacognitive monitoring and evaluation of their learning processes, it is not clear how this situation will improve. One of the many ways in which this goal could be achieved is through a specific technique called “exam wrappers.”

An exam wrapper (also known as a “cognitive wrapper”; Bowen, 2017) is a brief activity in which students complete a form to assess their recent exam performance, describe and reflect on how they prepared, and make a strategic plan for future improvement. This would typically be given to students upon receiving exam grades, with the goal to shift the focus from course content and exam outcome (grade) to the learning process itself. Since being introduced by Marsha Lovett in 2013, educators have been encouraged to use this tool to improve student metacognition and, ultimately, performance on exams and assignments.

There is surprisingly little well-controlled research on exam wrappers, and the several studies that have evaluated their impact are lacking in statistical power, internal validity, and/or generalizability. Raechel Soicher and Regan Gurung note this issue at the start of their 2017 article in which they report the results of an exam-wrapper intervention in introductory psychology. They compared an exam wrapper (modeled on Lovett, 2013) to both a “sham wrapper” condition in which students evaluate their incorrect answers and connect each to a relevant course topic, and also to a true control condition in which students simply reviewed their exams without explicit instruction. Results showed no differences among conditions in final grades (even when controlling for pre-intervention metacognition scores), nor on any of the exams, nor on metacognition subscale scores. The authors suggest that exam wrappers may be more successful when used across multiple classes, and that it may also help to make them more interesting and engaging for students. As I suggest below, perhaps having students complete the exam wrappers in the context of having learned about effective study strategies would also improve the benefit of implementing them after exams.

Another recent study, published in 2017 by Patricia Chen and colleagues, reported on outcomes from an exam-wrapper-type of activity called a “Strategic Resource Use” (SRU) intervention. Students in an introductory statistics course were randomly assigned to the SRU intervention or to a control condition that experienced many parts of the activity except for the focused metacognitive components. Importantly, this approach differs from that of traditional exam wrappers in that (1) it was self-administered and fully online; and, more importantly (2) there were both pre- and post-exam components. In the 7-10 days prior to taking the exam, all students completed an online survey in which they reported their predicted exam grade, motivation level, importance of achieving that grade, and confidence in reaching their performance goal. Those in the SRU condition also answered questions about the upcoming exam format, the types of resources available to them during preparation time, why each would be useful, and their plan for using each one. From a checklist of class resources, SRU students provided elaborated answers on usefulness and strategic planning. After the exam, students reported on which they had used, level of perceived usefulness, and how much self-reflection they had engaged in with regard to learning course material. Results showed that in comparison to the control condition, SRU students had higher course grades (about 1/3 of a letter grade), lower self-reports of negative affect toward exams, and higher perceived control over exam performance.

It is interesting that Chen and colleagues do not make the connection to the exam wrapper idea or literature. Both interventions described above have similar implementation and goals surrounding exams – to improve undergraduates’ self-regulated learning by focusing their attention on how they currently learn, how the quality and/or quantity of preparation map on to exam performance, and how they can use various strategies to improve for next time. Both interventions are based on the idea that highlighting the essential metacognitive processes of reflecting and adjusting supports student learning.

What to do with this mixed evidence and varying models for implementing this metacognitive “wrapper” tool? I have personally been using post-exam wrappers (modeled on Lovett) in my Cognitive Psychology course for several years. Though I have not collected empirical data on their effectiveness, based on student comments and my own observations I believe they help and plan to continue to use them. After considering Soicher and Gurung’s methods and results, I think that my implementation may be especially poised for single-course success because, unlike in the two studies discussed above, my exam wrappers are administered on the heels of learning about and engaging in practice with evidence-based learning strategies such as elaboration and frequent, effortful, and distributed (spaced) retrieval practice.

In addition to incorporating these elements into my course structure to provide students with multiple tools for durable learning, they also read the book “Make It Stick” (Brown, Roediger, and McDermott, 2014) early in the semester and engage in writing and peer discussion about effective ways to learn as described in my 2017 blog post Make It Stick in Cognitive Psychology. Thus, when my students complete the post-exam wrapper by reporting strategies they used, and those they will try to increase for future exams, they are doing so in a context of this metacognitive knowledge and accompanying motivation to learn. I am planning to add a pre-exam wrapper component, similar to the SRU model, the next time I teach this course, and given Chen et al.’s promising results, I hope it will even further support my students’ metacognitive development, learning, and, yes, course performance.

I explicitly communicate my perspective on exams to students, early and often: tests are learning events. By incorporating exam wrappers, I am reinforcing this message, and my students see that I care about their learning and my genuinely want them to improve. This also connects to a chapter in “Make It Stick” on the benefits of having what Carol Dweck calls a growth mindset – believing that intelligence is malleable and can be enhanced through practice and strategic effort. I encourage my students to adopt this mindset in multiple ways, and one way I can explicitly support this is to provide opportunities to learn from their experiences, including course exams.

Suggested References

Bowen, J. A. (2017). Teaching naked techniques: A practical guide to designing better classes. San Francisco, CA: Jossey-Bass.

Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick: The science of successful learning. Cambridge, Massachusetts: The Belknap Press of Harvard University.

Chen, P., Chavez, O., Ong, D. C., & Gunderson, B. (2017). Strategic resource use for learning: A self-administered intervention that guides self-reflection on effective resource use enhanced academic performance. Psychological Science, 28(6), 774-785. https://doi.org/10.1177/0956797617696456

Lovett, M. C. (2013). Make exams worth more than the grade: Using exam wrappers to promote metacognition. In M. Kaplan, N. Silver, D. LaVauge-

Manty & D. Meizlish (Eds.), Using reflection and metacognition to improve student learning: Across the disciplines, across the academy (pp. 18-52).

Soicher, R. N., & Gurung, R. A. R. (2017). Do exam wrappers increase metacognition and performance? A single course intervention. Psychology Learning & Teaching, 16(1), 64-73. https://doi.org/10.1177/1475725716661872

Personal Characteristics Necessary for Metacognition

By Lauren Scharff, Ph.D., U. S. Air Force Academy *

At my institution we have created the Science of Learning Team, a group of students who learn about the science of learning (including metacognition) and then lead seminars for other students who are hoping to improve their academic success. Additionally, as part of an ongoing scholarship of teaching and learning (SoTL) project, a small group of us (faculty and students) has assessed the efficacy of our various efforts to disseminate the science of learning to both faculty and students.

Students attending a seminar — Students attending a science of learning seminar

This past academic year I had the pleasure of working with Troy Mather, a senior who joined both the Science of Learning Team and the SoTL project effort as his capstone project. Below are some of his final reflections regarding his experiences helping develop his peers’ metacognition and learning skills. I believe they provide some great insights regarding the personal characteristics necessary for metacognition. He also shares some personal applications that many of us might use as a model as we work to develop metacognition in our students.

What is metacognition and why is it important?

My personal definition for metacognition is having the awareness to self- regulate your learning approaches through modifications or corrections. Having the awareness to identify what you need to work on or change gives you the opportunity to grow. It does require modesty and humbleness to look at yourself and be motivated to change something you see as an area for growth. If you are someone like me, who isn’t someone naturally gifted with academics, metacognition is a tool that you can use to guide your growth as a student and learner.

What is the biggest challenge to developing student metacognition skills?

The biggest challenge I see with developing student metacognition skills is the fact that this skill is largely correlated with maturity. Time is a limiting factor because developing self-regulation doesn’t happen overnight. This makes teaching metacognition hard because you can tell others the definition of the concept and why it is important, but you can’t make them internalize the importance or change their behaviors. However, I have seen that most students eventually figure it out with time and maturity.

How can we overcome this challenge?

Something I found to help students get on that track of appreciating metacognition is by providing some personal examples of ways I have self-regulated my learning approaches and made clear improvements. Students listen to those moments of success and often feel more willing to make changes or even become more aware of what they should work on. Sometimes this goes outside of the academic environment. For example, one of the ways I have been most impacted by metacognition is with my training to be selected for Special Tactics/ Combat Rescue following my graduation.¹
I told my students in our Science of Learning seminars that my training experience was a journey of self-reflection and deep accountability. Every day I had to have the self-awareness and honesty to identify my weak areas and do something about it. Some days I didn’t want to drown in the pool. Some days I didn’t want to run a marathon. And some days doing thousands of body weight exercises when I was already sore was a miserable thought. But, I pushed myself to do those things everyday because I knew if I didn’t, I wouldn’t reach my goal. I got a professional free- dive instructor and a track coach to help me with my training regimen. It was metacognition that allowed me to see areas to improve and reach out for resources.
With academics, students need to take advantage of all the resources they have in front of them. But, this requires self-accountability to make those identifications and be willing to put in that extra work. I told our students about my experience training for Special Operations because they hopefully saw someone with high ambitions and the willingness to put in the work. Every once in a while, learners need a motivational story to put them on track to accomplish their own goals. I have learned that metacognition is the start to achieving any level of greatness.

Using Troy’s Examples

Troy mentions humbleness and self-honesty as underlying characteristics of successful engagement in metacognition. That is not an aspect of metacognition that I have seen widely discussed, but it’s a great insight. It can be uncomfortable acknowledging aspects of our own efforts that have not been successful, and then examining them closely enough to come up with alternate strategies. This discomfort is especially strong if the alternate strategies appear to require more effort, and we’re not certain that they will lead us to success.

Many of our students face these uncomfortable moments on their path to become better learners. Perhaps we can help them through these uncomfortable barriers by more openly acknowledging the discomfort in facing one’s shortcomings, and letting students know that they are not alone in experiencing discomfort. Motivational stories such as the one Troy shared can help ease the resistance to being metacognitive. I’m sure we can all come up with a personal story or two that illustrate our own experiences as developing learners in some realm. Hopefully we can move past our discomfort in sharing our struggles in order to motivate our students to face their own struggles and self-regulate to move beyond them.

————————–

¹ Special Tactics/ Combat Rescue is an elite team within the Special Operations career field of the Air Force.

* Disclaimer: The views expressed in this document are those of the author and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.

Investigating Students’ Beliefs about Effective Study Strategies

By Sabrina Badali, B.S., Weber State University
Cognitive Psychology PhD student starting Fall ‘19, Kent State University

As an undergraduate, I became familiar with the conversations that took place after a major test. My classmates frequently boasted about their all-nighters spent reviewing textbooks and notes. Once grades were released, however, another conversation took place. The same students were confused and felt their scores did not reflect the time they spent preparing. My classmates were using relatively ineffective study strategies; most likely because they did not understand or appreciate the benefits of more effective alternatives.

Some of the most commonly reported study strategies include rereading a textbook and reviewing notes (Karpicke, Butler, & Roediger, 2009). However, those strategies are associated with lower memory performance than other strategies, such as testing oneself while studying, spreading out study sessions, and interleaving or “mixing” material while learning (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). Getting students to change their study habits can prove difficult. An effective way to start, perhaps, is getting students to change their beliefs about these strategies.

Before a learner will independently choose to implement a more effective study strategy (i.e. spreading out study sessions), they need to appreciate the benefits of the strategy and realize it will lead to improved performance. It seems this is often where the problem lies. Many students lack a metacognitive awareness of the benefits of these effective strategies. It is common for students to believe that strategies such as rereading a textbook or cramming are more beneficial than strategies such as testing oneself while learning or spacing out study sessions, a belief that does not match actual memory performance.

Researching Interleaving as a Study Strategy

This underappreciation of the benefits of these effective study strategies was something I recently investigated. In my research project, undergraduate participants completed two category learning tasks – learning to recognize different species of butterflies and learning artists’ painting styles. For each learning task, half of the butterfly species and half of the artists were assigned to the massed study condition. In the massed condition, all images of a category would be presented consecutively before moving on to the next species or artist. For example, all four images of one butterfly species would be presented back-to-back before moving on to images of the next species. The remaining half of the categories were assigned to the interleaved study condition. In the interleaved condition, images from a category were spread throughout the learning task and two images from the same category were never presented consecutively. For example, the first image of the “Tipper” butterfly may be shown early on, but the remaining three images would be distributed throughout the learning task such that participants viewed several other species before viewing the second image of the “Tipper”.

Images illustrating both massed presentation (left side - all butterflies are in the same category) and interleaved presentation (right side - the butterflies come from four different categories).

After completing these tasks, and completing a final memory assessment, participants were given a brief explanation about the difference between the massed method of presentation and the interleaved method. After this explanation, participants provided a metacognitive judgment about their performance on the study. They were asked whether they thought they performed better on massed items, interleaved items, or performed the same on both.

Misalignment of Evidence and Beliefs

I found that 63% of the participants thought they performed better on massed items, even though actual memory performance showed that 84% of participants performed better on interleaved items. There was a clear disconnect between what the student participants thought was beneficial (massing) versus what was actually beneficial (interleaving). Participants did not realize the benefits of interleaving material while learning. Instead, they believed that the commonly utilized, yet relatively ineffective, strategy of massing was the superior choice. If students’ judgments showed they thought interleaving was less effective than massing, how could we expect these students to incorporate interleaving into their own studying? Metacognition guides students’ study choices, and, at least in this example, students’ judgments were steering them in the wrong direction. This poses a problem for researchers and instructors who are trying to improve students’ study habits.

Using these effective study strategies, such as interleaving, makes learning feel more effortful. Unfortunately, students commonly believe it is a bad thing if the learning process feels difficult. When learning feels difficult, our judgments about how well we will perform tend to be lower than when something feels easy. However, memory performance shows a different pattern. When learning is easy, the material is often quickly forgotten. Alternatively, when learning is more difficult, it tends to lead to improved longer-term retention and higher memory performance (Bjork, 1994). While this difficulty is good for learning outcomes, it can be bad for the accuracy of metacognitive judgments. Before we can get students to change their study habits, it seems we need to change their thoughts about these strategies. If we can get students to associate effortful learning with metacognitive judgments of superior memory performance, we may be able to help students choose these strategies over others.

When teaching these study strategies, explaining how to use the strategy is a vital component, but this instruction could also include an explanation of why the strategies are beneficial to help convince students they are a better choice. Part of this explanation could address the notion that these strategies will feel more difficult, but this difficulty is part of the reason why they are beneficial. If students can accept this message, their metacognitive judgments may start to reflect actual performance and students may become more likely to implement these strategies during their own studying.

References

Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.). Metacognition: Knowing about Knowing (pp. 185-205). Cambridge, MA: MIT Press.

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58.

Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471-479.