Finding Your People

by Dr. Leah Poloskey, Assistant Clinical Professor, Department of Exercise and Rehabilitation Science, Merrimack College, and 

by Dr. Sarah Benes, Associate Clinical Professor, Department of Nutrition and Public Health, Merrimack College

(Post #1: Integrating Metacognition into Practice Across Campus, Guest Editor Series Edited by Dr. Sarah Benes)

How it all began . . .

Reflecting on the journey to having a Guest Editor spot with a mini-series of blog posts about metacognition with our colleagues from across campus was a great opportunity to reconnect to the power of community in this work. And it all began with a problem . . .

We had been discussing how challenging it was to engage students in our Health Science classes (Leah teaches in the Exercise and Rehabilitation Department and Sarah teaches in the Nutrition and Public Health Department). We decided to work together to investigate more deeply (rather than just dwelling on the challenge). We applied to host a Teaching Circle, which is an informal structure at Merrimack College that allows faculty and staff to come together around common interests. Teaching Circle facilitators are awarded small stipends for their time and effort in developing and running these opportunities. We believed that the Teaching Circle structure would provide a great opportunity for us to work within existing campus initiatives to enhance collaboration and engagement with faculty and staff across campus.schematic of three people facing each other with lightbulb being lit over their heads

Our first Teaching Circle was about student engagement. We ended up exploring mindset and the ways that mindset can impact engagement. We conducted a research study where we developed a tool that essentially is a measure of metacognitive states (Mandeville, et al., 2018). With this tool we learned how to assess a student’s self appraisal of their learning, which is a great opportunity to review a student’s intellectual development, mindset and metacognition. Now we had a way to assess these constructs, but what next?

We decided to apply for another Teaching Circle with a focus specifically on Metacognition. Our idea was approved and we were able to engage an even larger group of faculty, staff, and administrators from our academic support staff, to the psychology and business departments and more! Everyone in the group was interested in learning more about ways to support metacognition in our students in our various spaces. And this was the beginning of this blog post series!

What We Learned

Every meeting we had brought together a different group of people depending on schedules and availability. We had core folks who came each time and then a variety of others who came when they were able. Thinking about it now, we remember every meeting being exciting, dynamic, and invigorating.

We didn’t have set agendas and we didn’t have much reading or preparation (unless people asked for items to read). We really just came together to talk and share about our successes and challenges related to supporting students developing their metacognitive skills and to brainstorm ideas to try in our spaces. However, this opportunity for informal community gathering and building was a needed breath of fresh air. We always left energized for the work ahead (and we think the other participants did too!).

In fact, as a result of the Metacognition Teaching Circle, we embarked on a whole new project in which we used the MINDS survey (Mandeville, et al., 2018) at the beginning of the semester and then created “low touch” interventions to support metacognition and growth mindset depending on how students scored on the scale. From this we learned that many students are not familiar with concepts of metacognition and mindfulness, that many actually appreciated the tips and strategies we sent them (and some even used them!), and that students felt that more learning on these topics would be beneficial.

This then lead us to another study, this time examining faculty perceptions of metacognition which we were excited about because our experience suggested that it is likely that folks in certain settings or with certain backgrounds would be more familiar with metacognition and that faculty may not have the understanding or skills to teach metacognition in their courses. For faculty, it is so important to understand the idea of metacognition as it enables students to become flexible and self-directed learners. The teaching and the support of metacognition in the classroom is impactful. It allows students to become aware of their own thinking and to become proficient in choosing appropriate thinking strategies for different learning tasks. Unfortunately, this line of inquiry did not last long due to COVID 19 but we hope to pick this back up this year as we feel it is an important area that could be impactful for faculty and students.

While the research ideas and changes to practice are exciting and were impactful benefits of our Teaching Circles, one of our biggest takeaways was the reminder of the importance of finding others who are also doing the work. Sometimes on our campus, and we suspect it is the case at other institutions as well, we get siloed and often our meetings are with the same folks about the same topics. Being able to facilitate and participate in a cross-campus initiative about a passion topic was an amazing opportunity to meet new people, make new connections, gain different perspectives and create new ideas and strategies to try. We found many people doing great work with students on our campus across so many different departments and schools, and most importantly, found “our people” – people who you can go to when you are stuck, people who you can bounce ideas off of and collaborate with . . . we found our “metacognition people” (some of them at least).

While this was not a “new” idea or “cutting edge”, coming off a year in which we have been separated (in so many ways), we were reminded of the power of connections with others to maintain and sustain ourselves as academics and as humans. We wanted to share that in the guest series by not only showcasing some of the work that our colleagues are doing but also to remind readers to try and find your people . . . whether they are on your campus or off, whether you meet in person or virtual – or only via Tweets on Twitter . . . find the people who can help you maintain, sustain and grow your interest, skills, passion and joy!

We hope you enjoy reading the work of our colleagues and that it helps you on your journey.

References

​​Mandeville, D., Perks. L., Benes, S. & Poloskey, L. (2018). The Mindset and Intellectual Development Scale (MINDS): Metacognitive Assessment for Undergraduate Students. International Journal of Teaching and Learning in Higher Education, 30(3).


From Faculty to Chair: Lessons Learned

by Dr. Scott Santos, faculty member who became Chair of the Department of Biological Sciences in 2018; thus, he was thrust into leadership of the curricular redesign project.  

In the final post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Scott Santos shares his experience of moving from a faculty member in the department when the process of improving metacognition in the department began, to becoming chair and suddenly in a position to lead it. He also shares key lessons we learned over the course of the project and looks ahead to what’s next for the Biology department.

Learning about Metacognition

“Metacognition…Huh? What’s that?!” is what popped to mind the first time I came across that word in an email announcing that our department would be investigating ways to integrate it into courses at our 2017 Auburn University (AU) Department of Biological Sciences (DBS) Annual Faculty Retreat. Testimony to my naïveté on metacognition at the time comes from the fact that the particular email announcing the above is the first containing that specific word among 100,000+ correspondences dating back to 2004 when I started as a faculty member.

The email mentioning metacognition prompted me to spend a few minutes researching the word and, not surprisingly, discovering a wealth of internet resources. One of the most useful I found among these came from the Center for Teaching at fellow Southeastern Conference school, Vanderbilt University, where it was defined as “…. simply, thinking about one’s thinking” (Chick, 2015). I found this interesting since it reminded me of a recurring comment I have heard over the years amongst individuals who have successfully defended their Ph.D. dissertations, namely that one’s defense makes you realize how much you know about one particular area of knowledge while realizing how little you know about everything else.

The point that jumped out at me concerning this potential analogy was that, if it represented a genuine example of metacognition, it evolves in an individual over multiple years as they experience the trials and tribulations (as well as rewards and eventual success) associated with obtaining a terminal degree. Ambitiously, we were taking on the challenge of attempting to instill in early-career students an awareness and recognition of their strengths and weaknesses across the spectrum of learning, writing, reading, etc. As you can imagine, this was my first indicator that we had some work to do.  

Educating Our Department

So how did the DBS faculty at AU approach this seemingly daunting task of bringing metacognition “to the masses”?

Firstly, our previous departmental leadership had the foresight to start the process by having the retreat facilitated by highly-qualified individuals. Specifically, Dr. Ellen Goldey (currently Dean, Wilkes Honors College, Florida Atlantic University; formerly Department Chair, Wofford College, SC) and Dr. April Hill (Chair, Department of Biology, University of Richmond) were recruited as two nationally-recognized leaders involved in the National Science Foundation (NSF)-funded Vision and Change (V&C) Report (Brewer & Smith 2011) to conduct a workshop that included integration of metacognition into our curricula. This proved highly useful to having our faculty begin to wrap our collective minds around what metacognition was (and could be) along with how we might begin approaching its integration into our existing and future courses. This has been followed up by general faculty meetings, and subcommittee meetings such as those of the DBS Curriculum Committee, which have occurred at regular intervals to undertake this process. 

photo of Bio Dept faculty fall 2019
Fall 2019 Biology faculty retreat group photo. Scott Santos, front and center, giving the Shaka sign.

Overall, I am happy to report that we have made some significant progress in this area, including holding specialized workshops on the topic and discussing approaches to incorporate metacognitive prompts into midterms, finals, and surveys of undergraduate student research experiences that collect responses for future qualitative analyses, for integration of metacognition development and assessment into our budding ePortfolio initiative, and other activities (like this blog series). However, these modest successes have not come without challenges: as our “metacognition massacre” experience taught us, it takes significant levels of time and energy for such efforts to come to fruition and to seed and foster support for these efforts among the faculty charged with bringing metacognition into the classroom.  

Key Insights

It has now been several years since AU DBS started our initiatives with metacognition, and during this time I have transitioned from an individual faculty member “in the trenches” to Chair of the department and thus charged with “leading the troops.” While I would be well-off financially if I had received a nickel for every time I have been offered “congratulation, and condolences” in the year and a half since becoming Chair, it has given me a new and different perspective on our metacognitive efforts:  

  1. First, a passionate and dedicated team is needed for initiatives like this to prosper and we, as a department, have been fortunate to have that in the form of our AU colleagues who are also contributing blogs in this series. Importantly, they belong to multiple units outside DBS, thus bringing the needed expertise and perspective that we lacked or might miss, respectively. We are greatly indebted to them, and departmental chairs and heads interested or intending to start similar initiatives would be wise to establish and cultivate such collaborations early in the process. It is very helpful to have expert advice when tackling issues that are unfamiliar to most of the faculty.
  2. Second, working with your faculty on understanding what metacognition is, along with defining expectations and assessment for initiatives around it, are paramount for your department’s immediate success with implementing activities from such efforts. In our case, the fact that many AU DBS faculty wrestled with the concept of metacognition meant that we had to invest more time for calibration before discussion could move forward.
  3. Third, the significance of solicitating undergraduate student participation during the development and implementation stages of the process should not be undervalued since they are the constituents who our efforts are ultimately targeting and thus deserve a voice at the table. Although our posts in this series have highlighted inflection points for the faculty as we moved our curriculum toward more metacognition, it is critical to note that we involved students as partners throughout the process. Some strategies we used include organizing student focus groups led by facilitators outside the department, conducting surveys, and inviting students to some meetings and department retreats.

Importantly, this should not be considered an exhaustive list and instead should serve as a general guide of issues to consider from someone who has had an opportunity to both witness and participate in the process from the departmental faculty and leadership perspectives.  

Looking Toward the Future

What does the future hold for AU DBS when it comes to metacognition? On one hand, we will continue in the short-term to implement the initiatives described above while being opportunistic in improving them, which we consider to be a strategy consistent with the current stage of our efforts to develop metacognitive abilities in students enrolled in our programs. On the other hand, the long-term forecast, at least from the departmental standpoint, is more amorphous, with reasons for this including our need to involve a large number of newly recruited faculty. We look forward to new directions and possibilities as we learn about new strategies from our colleagues, though we recognize the need to balance and maintain synergy between departmental undergraduate and graduate programs in the face of limited resources.

Finally, a key element for metacognition highlighted by Vanderbilt’s Center of Learning is “recognizing the limit of one’s knowledge or ability and then figuring out how to expand that knowledge or extend the ability.” Given that, I would like to think that Auburn’s Department of Biological Sciences itself is attempting to be metacognitive in its approach to preparing and fostering metacognition in students, and it will be interesting to see how our current efforts evolve in the future.  

References:

Chick, N. (2015). Metacognition: Thinking about one’s thinking. Vanderbilt University-The Centre for Teaching. 

Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science, Washington, DC


Assessing Metacognition: A Plan for Learning Improvement

In the fourth post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Lindsay Doukopoulos describes the goals and outcomes of the spring meetings of the Biology curriculum committee that were led by Biggio Center and the Office of Academic Assessment. Namely, they sought to create a questionnaire that would be given to all graduating students that would allow them reflect on their learning over the course of their academic career and to create a rubric to measure the quality of metacognition in their responses.

by Dr. Lindsay Doukopoulos, Assistant Director of the Biggio Center for the Enhancement of Teaching and Learning. In collaborating with the Office of Academic Assessment on the Learning Improvement Initiative, she leads faculty development initiatives designed to connect faculty with strategies, resources, and partners to support their teaching. 

In Spring of 2019, Katie Boyd and I led four meetings with Biology’s curriculum committee with the two-part goal of producing a set of metacognitive-reflection questions to be completed by every graduating student in their capstone course and a rubric to assess the quality of metacognition evidenced in the reflections.  

Developing the Rubric

In the first workshop, our goal was to help faculty unpack the definition of metacognition into categories and decide how many levels or standards to maintain within the rubric. In other words, we were hoping to fill in the x and y axes of the Metacognition rubric.

To facilitate this discussion, we brought two rubrics designed to measure metacognition. One came from the General Learning Outcome site of Cal State University-San Bernardino (CSUSB). The other came from the AAC&U Value rubric on Lifelong Learning, specifically, the two elements called Transfer and Reflection. Both rubrics appeared to offer valuable ways of assessing the metacognition evident in a written reflection. Rather than choose one or the other, we combined the two categories (rows) of the AAC&U Value rubric and the three categories (rows) of the CSUSB rubric. We also decided to use four standards of quality (columns) and discussed terminology resulting in: Beginning or N/A, Emerging/Developing, Mastery, and Exceeding Expectations.  

photo of a curriculum meeting
Spring 2019 Biology undergraduate curriculum meeting number 1: creating a rubric to assess metacognition using the newly defined and approved SLO 6.

In the second workshop, our goal was to fill in the performance criteria or behavioral anchors that would make up the rubric. After much discussion, we again decided to leave the rubric big and pilot it in our next meeting to determine whether the AAC&U elements or the CSUSB elements would be preferable.

In our third workshop, piloted the rubric by we scoring a packet of student reflections that had come out of the Biology undergraduate research capstone course the previous year. In practice, the faculty found the two elements of the AAC&U rubric easier to apply and more valuable for differentiating between the quality of metacognition in the student responses. Thus, we reduced the final rubric to those two elements.

chart showing a metacognition rubric for biological sciences
This is the rubric to assess metacognition that came out of Biology’s spring curriculum committee meetings with Biggio Center and Academic Assessment.

Developing the Reflection Questions

In the final workshop, our goal was to draft and finalize questions that would be given to sophomores and seniors in the program. These questions would parallel those already being used in the undergraduate research capstone course. These are the questions the committee created:

  1. What has been your favorite learning moment in your major? Please describe it in detail and explain why it was your favorite.
  2. What were the most useful skills you learned in your major and why?
    1. Regarding the skills you listed in question 2: how do you know you learned them? Please provide specific examples.
    2. How do you plan to apply these skills in future courses or your career?
  3. As a student, what could you have done to learn more? If you could go back in time and give yourself advice, what you say?
  4. Evaluate your capacity to design an experiment and generate hypotheses. Please provide specific examples of aspects of the scientific process you’re most and least confident about.
  5. Reflect on your view of science. How has your participation in your Biological Sciences major changed your view of science, if at all? Please provide specific examples.
  6. Reflecting on your learning journey, what do you value most about your major curriculum (i.e. the courses you took and the order you took them in)?

This question-writing process concluded the initial phase of the Learning Improvement Initiative as it led to the creation of the instrument the department will use to gather baseline data on the metacognition SLO. Moving forward, all students (roughly 75 majors per year) will complete the questionnaire during their capstone course and the curriculum committee will lead assessment using the rubric we created.

The goal is to have every student scored on the rubric every year beginning with baseline data collection in spring 2020 with students who have not experienced the “treatment” conditions, i.e. courses redesigned by faculty to improve metacognition. Over time, we expect that the faculty development workshops around transparent assignment design, reflective writing assignments, and ePortfolio pedagogy will result in graduates who are more metacognitive and data that reflects the learning improvement.  


Re-Defining Metacognition: Generating Faculty Engagement

In the third post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Chris Basgier describes the workshops series led by Office of University Writing and the Biggio Center that helped the department redefine metacognition in such a way that they felt like they could understand it, teach it, and assess it. He also unpacks the value of the new definition and points to the work ahead as Biology embraces ePortfolios as part of a pedagogical strategy to increase metacognition in their students.

by Christopher Basgier, Associate Director of University Writing 

In the fall semester of 2018, Lindsay Doukopoulos and I had the opportunity to guide faculty from Auburn University’s Department of Biological Sciences (DBS) through a series of workshops devoted to metacognition. These workshops were a direct response to the “metacognition massacre” that had occurred at the August 2018 faculty retreat, as Dr. Robert Boyd recounted in the first blog post in this series.

Photo of Fall 2018 Biology faculty workshop number one: using the TILT Higher Ed transparent assignment design framework to improve metacognition.
Fall 2018 Biology faculty workshop number one: using the TILT Higher Ed transparent assignment design framework to improve metacognition.

Essentially, DBS faculty were uneasy with the definition of metacognition contained in the department’s student learning outcome (SLO), and unsure how to implement metacognitive activities in their courses. Working together, Lindsay and I decided to use these workshops to introduce faculty to the principles of transparent assignment design, offer guidance on integrating reflective writing into courses, and work with them to redefine the metacognition SLO in more familiar terms. 

Transparent Design

We began with transparent assignment design and reflective writing—rather than the SLO—to generate faculty engagement in metacognition. With concrete such strategies for promoting metacognition under their belts, we decided, faculty would be more invested in redefining the SLO and more willing to commit to aligning their courses to that outcome.  

Lindsay led the effort to introduce transparent assignment design to DBS workshop participants. According to Mary-Ann Winkelmes with TILT Higher Ed (2016), transparent assignment design invites faculty to clarify how and why students are learning course content in particular ways. Transparently designed assignments include   

  1. The assignment’s purpose, including the skills they will practice and the knowledge they will gain  
  2. The task, including what students will do and the steps they should take to complete the assignment  
  3. Criteria for success, including a checklist or rubric and examples of successful student work  

From our perspective, transparently designed assignments can promote metacognition. They make explicit what is often implicit in course assignments, and they help students see how a given assignment fits within the larger context of a course, and even a curriculum. The best designed assignments show students how to draw on what they already know, and help them imagine future implications of their work.  

We gave faculty ample time during the first workshop to consider how they would revise one or more assignments using the transparent assignment design framework, but we also knew that students needed to take an active role in their learning if they were to enhance their metacognitive capabilities. Therefore, I led a second workshop on reflective writing.   

Reflective Writing Component

In Auburn’s Office of University Writing, where I work as Associate Director, we spend a lot of time introducing principles of reflective writing to faculty, namely because we are in charge of the ePortfolio Project, which is Auburn’s Quality Enhancement Project required for accreditation in the SACSCOC.

The ePortfolios we support are polished, integrative, public-facing websites that students can use to showcase their knowledge, skills, and abilities for a range of audiences and purposes. A key component of ePortfolios, reflective writing is a metacognitive practice that invites students to articulate learning experiences, ask questions, draw connections, imagine future implications, and repackage knowledge for different audiences and purposes. After introducing DBS faculty to various levels of reflective writing, I gave them time to develop a reflective writing activity that would support a project or experience already in play in the courses.   

Our hope in these first two workshops was to give DBS faculty practical tools for promoting metacognition in their courses that would not require wholesale course redesign. Transparent assignment design and low-stakes reflective writing are fairly easy to implement in most course contexts.

Redefining the Metacognition Learning Objective

Our third workshop required more intellectual heavy lifting, as it focused on redefining the metacognition SLO. The original metacognition SLO read as follows:  

Students will develop metacognitive skills and be able to distinguish between broad categories of metacognition as applied to their major. In particular, they will distinguish between foundational (i.e., knowledge recall) and higher order (i.e., creative, analysis, synthesis) metacognitive skills.  

The trouble with this definition is that it seems to require students to be able to define different kinds of metacognition (which is difficult enough for faculty), rather than put different kinds of metacognition into practice, regardless of whether or not they can name the metacognitive “categories” they are using.

As an alternative, I turned to research by Gwen Gorzelsky and colleagues, scholars in writing studies who developed a taxonomy of kinds of metacognition. In their framework, the richest form of metacognition is constructive metacognition, which they define as “Reflection across writing tasks and contexts, using writing and rhetorical concepts to explain choices and evaluations and to construct a writerly identity” (2016, p. 226). 

Attracted to the notion that metacognition involves reflection on choices and the construction of identity, Lindsay and I tried our hand at a revised definition:  

Metacognition is defined as the process by which students reflect on and communicate about their role in learning. Reflection and communication may include: 1. students’ choices made in response to the affordances and constraints on learning, and/or 2. students’ evaluations of the success of such choices, particularly across tasks and contexts. Ultimately, these activities should help students develop and articulate identities as scientists.  

Our goal in composing this definition was not to suggest to DBS faculty that it was the right one, only that alternatives were possible. During the final workshop, we asked them to review the original SLO as well as our alternative, and then apply some “critical resistance” to each by reflecting on which terms or ideas made sense, which did not, and what language they might like to include. After much discussion, the group developed a revised SLO:  

Students will develop their metacognitive skills. Metacognition is defined as the process by which students reflect on and communicate about their role in learning. Reflection and communication may include: 1. Awareness of choices made in response to the opportunities (i.e., homework, office hours, review sessions) and constraints (i.e., challenging problems, short time frames) on learning, and/or; 2. Evaluation of the success of such choices, particularly across tasks and contexts. Ultimately, these activities should help students develop and articulate their science knowledge and its value to their professional and lifelong learning goals.   

This definition includes some key changes and additions. It eliminates jargon like “knowledge recall” and “affordances” in favor of more accessible language like “opportunities,” which are further defined in parentheses. Faculty also pushed back on the idea that all students should develop identities as scientists. A great number of students who take DBS courses plan to go into medical fields, so instead, they wanted to put the emphasis on science knowledge, a much more portable focus than science identity. They also added the notion of professional and lifelong learning goals to acknowledge the varied contexts in which their science knowledge might be relevant.   

In the end, our metacognition workshop was a success: the department approved the new definition in December 2018, and many commented on how much clearer and easier to implement and assess it appeared. But our work is not done. Faculty still need to integrate metacognition throughout the curriculum—or at least in courses where it is feasible. The department has agreed that ePortfolios are an effective vehicle for doing so.

ePortfolios to Support Implementation

DBS had joined the ePortfolio Cohort (the group of departments and units committed to implementing ePortfolios) in 2017, and have been working steadily on implementation. Valerie Tisdale, the department’s academic advisor, began the effort to introduce ePortfolios in BIOL 2100, a professional practice course for undergraduate biology majors, in fall 2018. Most recently, in spring 2019, DBS faculty applied for and were awarded with a grant to support an intensive summer workshop to further the integration of ePortfolios in support of metacognition and written communication. My colleague Amy Cicchino and I met with three department members—Lamar Seibenhener, Joanna Diller, and Valerie Tisdale—for four weeks in summer 2019. Utilizing the resources of the ePortfolio Project, the departmental team developed a host of materials for a new, required course that asks students to complete their final ePortfolios during their senior year. 

In the interest of transparent assignment design, they also created an ePortfolio “roadmap” that would help DBS majors understand what an ePortfolio is, why it is important for students in the sciences, and where in the curriculum they might encounter artifacts that could be used as evidence of their knowledge, skills, and abilities. The department approved the new course and completed the roadmap at a retreat in late 2019.

At this point, we are awaiting university-level approval of the new course. In the meantime, we are also planning workshops for DBS faculty on designing meaningful assignments that can be used as ePortfolio artifacts. Taken together, these efforts will help DBS support metacognition through ePortfolios in the years to come.  

References

Gorzelsky, G., Driscoll, D. L., Paszek, J., Jones, E., & Hayes, C. (2016). Cultivating constructive metacognition: a new taxonomy for writing studies. Critical transitions: Writing and the question of transfer, 215

Winkelmes, M. A., Bernacki, M., Butler, J., Zochowski, M., Golanics, J., & Weavil, K. H. (2016). A teaching intervention that increases underserved college students’ success. Peer Review, 18(1/2), 31-36. 


Project Beginnings

In the second post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Katie Boyd describes the activities the year prior to the Metacognition Massacre. These early activities started with the Learning Improvement Initiative that marked the beginning of the collaboration between Biology, Office of Academic Assessment, and the Biggio Center. She also describes how the initial definition of the metacognition learning outcome came about, how the department came to a greater understanding of metacognition, and how that understanding prompted a redefinition of what they believe metacognition is and should be within their context.

by Katie Boyd, Associate Director of the Office of Academic Assessment 

Luckily, the work of the Department of Biological Sciences to increase their graduating students’ metacognitive skills did not simultaneously begin and end with the “Metacognition Massacre” of 2018! If we back up just one year, the department was coming off of a strong fall faculty retreat and was ready to turn attention to the thoughtful examination of their curriculum and the knowledge, skills, and abilities expected of all students graduating from their program(s).    

In 2017, each undergraduate degree program in the Department of Biological Sciences (Marine Biology, Microbial, Cellular, & Molecular Biology, and Organismal Biology) had two (2) student learning outcomes and they addressed critical reading, information literacy, and communication skills. Metacognition had only just entered the conversation: it had not been a thoughtful component of the curriculum nor was it a learning outcome for their graduating students. The Department of Biological Sciences needed help. 

Partnering with the Teaching & Learning Center

Enter Auburn University’s Biggio Center for the Enhancement of Teaching and Learning, and Office of Academic Assessment.  That fall semester the two offices joined forces to support programs interested in evidencing learning improvement and jointly issued a request for learning improvement proposals.  

The learning improvement initiative was a way for programs to demonstrate a positive impact by showing how investment in innovative curricular experiences could lead to the improvement of student learning.  The Biggio Center and Office of Academic Assessment wanted to help programs evidence this improvement.  Of note, most departments redesign their curriculum too infrequently or do not have data to inform their curricular redesign, thus delaying their ability to showcase the improved preparedness of their graduates. We anticipated that the joint support of a teaching and learning center AND an assessment office would provide programs with many benefits, such as:   

  • A streamlined approach to aligning assessment processes with curricular innovation(s)  
  • The possibility of improving their students’ learning  
  • Strengthened program reputation  
  • Faculty satisfaction with process and outcome(s)  
  • Demonstrated good stewardship of departmental/college resources  
  • Opportunity for presentation/publication  

Biological Sciences submitted a proposal asking for support to define, measure, and improve metacognition amongst their graduates and they were chosen as one of six programs to participate in the inaugural cohort of learning improvement teams. Their specific reasons for choosing this outcome effort for their Learning Improvement Project are outlined below: 

  1. Metacognition was an element in the Action Plan developed by departmental  representatives at the PULSE Institute in June 2016. It was selected for the Action Plan  because it was a neglected element in our curricular planning.  
  2. This SLO was a new one on the department-wide list of SLOs, and of all the SLOs was the one with which faculty were least familiar. Specifically, the program felt they would need the most assistance integrating that into their degree programs. 
  3. A final reason was the hope that working with the Office of Academic Assessment and Biggio Center on improving students’ metacognition would eventually provide a model by which Biological Sciences could plan and implement curricular changes for their other SLOs.   

Writing the Learning Outcomes

Thus began the learning improvement project and, throughout the Fall semester, the Office of Academic Assessment facilitated a number of Biological Sciences curriculum committee meetings to re-write all of the department’s student learning outcomes (SLOs). The committee made incremental progress with bi-weekly meetings led by the Department’s intrepid chair.  The department chair quickly led the committee to write six of the seven student learning outcomes, but conversation continued around the metacognition outcome. 

Photo: Bob Boyd showcases Biology's Learning Improvement Project at the year one celebration event hosted by Biggio Center and Academic Assessment in fall 2018.
Bob Boyd showcases Biology’s Learning Improvement Project at the year one celebration event hosted by Biggio Center and Academic Assessment in fall 2018.

A number of committee members advocated for the importance of metacognition and reflection and admitted to embedding reflection components into weekly lectures and/or assignments.  On another hand, the former department chair  advocated for a definition that would be easily measurable and liked the idea of students being able to identify the level/type of learning being assessed in specific types of questions on exams or similar instruments (knowledge, comprehension, application).  Bloom’s taxonomy drove a lot of this conversation. 

Eventually, the committee finalized a metacognition SLO (6) and completed their list of seven department-wide SLOs (8 or 9 if you include major-specific outcomes).  At the time, the metacognition SLO was defined by the curriculum committee as:  

Students will develop metacognitive skills and be able to distinguish between  broad categories of metacognition as applied to their major. In particular, they willdistinguish between foundational (i.e., knowledge recall) and higher order (i.e., creative,analysis, synthesis) metacognitive skills. 

The list of outcomes was shared with all program faculty during a fall faculty meeting and they voted to accept the list as the new set of outcomes.  There were few questions regarding the outcomes during this meeting.  However, I think we can all agree that this is pretty typical when these sorts of items/topics are brought up in faculty meetings.   

Creating the Curriculum Map

A secondary goal of the curriculum committee was to draft a curriculum map aligning the new student learning outcomes with the required courses in each of the three undergraduate curricula.  The first few meetings allowed the committee to finalize the list of classes they wanted to include in the map and a subsequent discussion about how accurate a curriculum map would be when drafted by a subset of the faculty.  The curriculum committee entered the curriculum mapping conversations with some apprehension because the faculty in the room did not represent or teach all of the courses within the curriculum map. 

Eventually, it was decided that they would draft an aspirational curriculum map in which the ideal alignments would be suggested and discussed in a future faculty retreat. When it came to the metacognition outcome, the committee strongly felt as though it should be covered in each required course and that each course truly should be contributing to the students’ lifelong learning.    

Starting to Consider Assessment

With a set of student learning outcomes agreed upon, and a drafted curriculum map, the learning improvement conversation finally began to move towards assessment and measurement. Essentially, there needed to be a way to evaluate whether students were thinking about thinking and knowing about knowing. Enter the Office of University Writing.  It was at this point that Biological Sciences seriously considered ways in which ePortfolios could be used to both teach and assess metacognition.

Initial conversations targeted ePortfolios as a way to encourage reflective writing and simply “house” student assignments. This idea has blossomed and become much more than a data warehouse, and Chris Basgier (Office of University Writing) will expand on this in the next blog post.     This brings us to the Fall 2018 faculty retreat, which allowed for a guided and thoughtful discussion around each outcome and the aspirational curriculum map.  It was this thoughtful discussion that led to the very effective massacre of SLO 6, ultimately pointing to the need for a better definition of metacognition as a learning outcome. 


“The Metacognition Massacre”

In the first post of “The Evolution of Metacognition in Biological Sciences” guest series, Dr. Bob Boyd reflects on 2018 faculty retreat where Biology faculty rejected responsibility for teaching metacognition in their courses. He also shares where and how Biology’s journey to learning improvement around metacognition began.

By Robert Boyd, Professor of Biological Sciences and former Undergraduate Program Officer for Department of Biological Sciences (DBS). Currently, Associate Dean for Academic Affairs, College of Sciences and Mathematics 

My most memorable moment regarding metacognition occurred at a departmental faculty retreat in August 2018, right before the start of Fall Semester. Before this retreat, our departmental Curriculum Committee had created an “aspirational” curriculum map that purported to show which required courses addressed our brand-new list of eight or nine Student Learning Outcomes (SLOs) for each of the three majors in our department.

The Massacre

Metacognition, our new SLO 6, was selected as being a part of every required course. At the retreat, breakout groups were assigned to discuss and describe some aspects of several SLOs (one SLO per group, including a group assigned to “metacognition”) and put their ideas on a flipchart. When all the breakout groups reported, the metacognition group presented a blank flipchart page and said that they had been unable to decide what metacognition was.

Later during the retreat, when we discussed our “aspirational” curriculum map to convert it into a map that showed which SLOs were actually addressed in our core classes, almost all the checkmarks for metacognition were removed from the map. We asked faculty to place Post Its over the SLOs that they didn’t feel like their courses needed to address. In my mind, that retreat was a metacognition massacre. It showed that we needed to do some serious work to define that SLO as well as decide how to integrate it and measure it in our curricula.

Photo of a chart showing a curriculum map from a faculty retreat

Image 1: Biology’s ideal curriculum map presented at the 2018 Retreat. Faculty used slips of pink paper to indicate rejection of an SLO they didn’t think their individual course (left hand column) addressed. SLO 6, metacognition, was almost entirely stricken from the curriculum.

This blog series will present my department’s work on metacognition, mainly focusing on how we have proceeded since the memorable metacognition massacre at that faculty retreat. But I want to take some time now to set the stage by describing my department and some of our work prior to that retreat.

Setting the Stage for the Metacognition Massacre

Auburn University is a land-grant school with about 30,000 students, and has recently achieved the status of a Carnegie R1 institution (meaning that research is an important part of our mission). My department of 43 faculty is a Biological Sciences department, and our courses are vital to the university’s educational mission as well. As evidence of this, in an academic year we teach about 45,000 student credit hours. 

This new outcome effort began in January 2016, when one of our faculty, Jason Bond, became Chair and encouraged us to review our curricula, something that had not been done since 2008. Coincidentally, also in January 2016, our department was invited to participate in an NSF-funded Institute at Wofford College in South Carolina designed to help us begin the process of reviewing and revamping our programs.

In June 2016, a small departmental team attended the retreat which was focused on a report by the American Association for Advancement of Science (AAAS) on undergraduate biology education in the US. The report, entitled “Vision and Change in Undergraduate Biology Education: A Call to Action” (referred to as V&C below) and available from this link (https://live-visionandchange.pantheonsite.io/wp-content/uploads/2013/11/aaas-VISchange-web1113.pdf), pointed out that undergraduate biology education needed reform and the workshop involved assessing our department and its curricula.The assessment used a rubric that listed “Student Metacognitive Skills” as one of the ten elements evaluated, with an exemplary department described as “Instructors regularly integrate practice of effective metacognitive strategies within assignments. Most students become adept at reflecting upon, and improving, their own learning and coaching their peers.”

To begin the work of moving as a department from having no outcomes related to metacognition to one that placed it squarely in the SLOs for all of our programs, we held a retreat in 2017 which focused on High Impact Practices (HIPs). This retreat was facilitated by two nationally known educational leaders: Dr. Ellen Goldey (Dean, Wilkes Honors College, Florida Atlantic University) and Dr. April Hill (Chair, Department of Biology, University of Richmond). Faculty engagement at this event was strong and led us to begin the work of formally committing to a curriculum that would address metacognition as an outcome of our undergraduate programs.

In the spring of 2018, we held faculty meetings to introduce V&C concepts and ask the faculty in each of our three majors to evaluate our programs. In every case we decided we were at a “Beginning” stage. According to the V&C rubric, this means “Rarely are students encouraged to reflect on their learning strategies and skills. Study strategies, when discussed, may not be specifically geared to STEM learning or the particular student’s needs.” These meeting led to the 2018 Faculty Retreat described earlier which showed us how challenging it would be for us to understand and embrace our metacognition SLO. 

Citations  Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science, Washington, DC.


Changing Campus Culture with the Ace-Your-Course Challenge

In the final post of the guest blog series on “Working with Faculty to Promote Metacognition,” Dr. Eric Kaldor discusses lessons learned from the implementation of a campus-wide metacognition program inspired by Saundra McGuire’s work. The associated research project was awarded the Robert J. Menges Award for Outstanding Research in Educational Development by the Professional and Organizational Development (POD) Network.

by Eric Kaldor, Ph.D.; Associate Director, Sheridan Center for Teaching & Learning, Brown University

For many faculty members, the “fact” that some students are just not capable of college-level learning remains part of the taken-for-granted assumptions embedded in the culture of disciplines and campuses. Despite significant efforts to share insights from the scholarship on metacognition and growth mindsets (e.g. Doyle & Zakrajsek, 2013; Dweck, 2016; McGuire, 2015; Nilson, 2013), campus cultures are slow to change, and fixed mindsets continue to dominate many institutions. This post describes efforts to change the culture at the University of Rhode Island, the communication strategy we used, and some lessons learned.

With approximately 14,000 undergraduate students and 1,000 full and part-time faculty, the University of Rhode Island is a challenging setting to advance culture change. Our story began with a conversation with Melvin Wade, former Director of the Multicultural Student Services Center (MSSC). I was working in the Office for the Advancement of Teaching & Learning (ATL) and planning for Saundra McGuire to visit our campus. I was particularly concerned to fill our 1,000-person auditorium with students for her “Metacognition is Key” workshop. When I asked for his advice, Melvin insisted we must ensure her visit had a lasting impact on our campus. Toward this end, we assembled a group of professional staff and graduate students from ATL, the MSSC, the Academic Enhancement Center, First-Year Programs, and Professional Advising. Over a series of conversations, this informal group conceived of something we came to call the Ace-Your-Course (AYC) Challenge. We assumed we would only run the AYC Challenge once as a companion to Dr. McGuire’s workshop. Instead, a snowstorm gave the Challenge a much longer life.

Building on the McGuire Model

We designed the AYC Challenge to extend students’ metacognitive experience and reflections beyond Dr. McGuire’s workshop. We developed the AYC Challenge as four weekly self-assessment surveys (for detailed description see Kaldor & Swanson, 2019) to create additional metacognitive experiences (Flavell, 1979) by encouraging students to:

  1. Test learning strategies relevant to them individually.
  2. Engage in key practices for metacognitive reflection: observation, description, evaluation, and action planning.
  3. Feel part of a larger community working to grow as learners.

When a snowstorm postponed Dr. McGuire’s visit to the next semester, our multi-unit team led her workshop twice using slides and talking points from her book (McGuire, 2015) and invited students to participate in the AYC Challenge. Of the 240 students attending a workshop, 50 completed all four weeks of the challenge. After we shared the positive results from our pilot with faculty members, many encouraged their students to attend Dr. McGuire’s rescheduled workshop in September 2017. Some went further and agreed to share grade data as part of an IRB-approved study to examine how participation affected grades. We specifically identified a set of gateway science courses from Chemistry, Biology, and Nutrition and Food Sciences that have large enrollments of first-year students.

Over 1,000 students attended Dr. McGuire’s workshop with some in remote viewing locations, and 202 of those completed the second AYC Challenge. The self-reported results for this larger group were strikingly similar to those from students in the pilot AYC Challenge when we led the workshops. Holly Swanson and I analyzed final grades for 979 students in the eight gateway science sections (347 attended the workshop and of those 55 completed the challenge) using OLS regression with controls for several predictors of academic performance including high school GPA and exam 1 z-score. Compared with their peers who did not attend the workshop or participate in the challenge, attending the workshop and completing the AYC Challenge was associated with a final course grade half of a letter grade higher (Kaldor & Swanson, 2018).

Inclusive and Extensive Communication

Much of our success originated from a spiral of communication that grew outwards from a core group of professional staff and graduate students who became involved in planning for Dr. McGuire’s originally scheduled visit. Our colleagues working in various student support services helped develop a plan to reach students and motivate them to attend the workshop and participate in the challenge. These colleagues advised us on when to hold the workshop, how to market our efforts, and what kinds of messages would appeal to students.

One critical piece of advice was that students were more likely to attend if instructors offered extra credit. In the faculty development office, we knew that instructors of large enrollment courses would only offer extra credit if it did not add significant work. Using google forms, a mail merge add-on, card swipe readers, and course rosters, we developed a system for students to pre-register, receive reminder emails, and swipe their id cards after the workshop. With this system in place, instructors for over 30 courses received a list of student attendees within a week of the workshop.

To nudge students who attended the workshop to start the Challenge and complete all four weeks, we used two techniques. First, students were told that completing all four weeks would make them eligible for a drawing for ten $100 gift cards to the campus book store. Second, we started the Challenge at the end of the workshop with students selecting one or more strategies to try on a Google form at the end of the workshop.

photo of Ace Your Course Challenge winners
Four of the ten winners of a raffle for students who completed the Ace Your Course Challenge.

The next spiral outwards involved engaging more faculty in a conversation on the powerful ways they could help their students learn. Prior initiatives that had promoted Dweck’s (2016) insights on growth mindsets had primed many faculty and staff for these conversations. Specifically, they wanted to know what else they could do beyond promoting a growth mindset, and a metacognitive approach to learning strategies offered them concrete answers.

In addition, faculty members who had moved away from fixed mindsets about who could succeed in their courses shared their insights on how to approach their still skeptical peers. We developed a strategy of presenting quantitative data alongside student voices to describe the student experience (examples are available here: https://web.uri.edu/atl/ace-your-course-challenge/). Initially, our quantitative data was limited to student self-reports. With the benefit of a snowstorm, we had the chance to organize an IRB approved research project to answer important questions that skeptics raised.

As we shared this data on campus, we were asked to try different permutations of the Metacognition Workshop plus AYC Challenge in two different settings – a support program for conditionally readmitted students and two gateway chemistry courses. In addition, we were asked to offer workshops for professional staff and faculty so they could include McGuire’s approach in their programs and courses.

One of the most successful workshops, “Teach Your Students How to Learn in 50-minutes” provided an annotated version of Dr. McGuire’s slides with breakout discussions about the key messages to motivate students. This led many instructors to experiment with including different elements of her metacognitive approach to learning strategies into their courses.

Some Lessons Learned and Suspected

Each AYC Challenge has generated new data and insights into the potential for URI students to make significant gains in their metacognition. This new data has generated new conversations, which have led to variations on the McGuire workshop and/or the AYC Challenge. This has been a fruitful if unintended process.

Our skeptical internal voice continues to ask how we could nudge more students into participating. We noticed lower participation rates for students from historically marginalized groups in our gateway science course study. This led us to experiment with embedding the workshop plus challenge into courses, but our early experience raised many concerns around overloading instructors and maintaining fidelity with the core AYC challenge experience.

In a promising next iteration, my URI colleague Michelle Fontes-Barros has suggested a partnership with student organizations and clubs, particularly STEM affinity groups for students from historically marginalized groups. Convinced of the value, a student group might sponsor a workshop in a regular meeting space. Student leaders might promote peer commitments to complete the AYC Challenge. Past AYC Challenge participants might help present the workshop and send messages during the Challenge to encourage persistence. This next iteration has the potential to be much more student-centered, but it will be important to critically evaluate the student experience and share results with the wider university community to energize the campus conversation on metacognitive development.

Doyle, T., & Zakrajsek, T. (2013). The New science of learning: How to learn in harmony with your brain. Sterling, VA: Stylus Publishing.

Dweck, C. S. (2016). Mindset: The new psychology of success (Updated Edition). New York, NY: Ballantine Books.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34(10), 906–911.

Kaldor, E., & Swanson, H. (2018, November). A campus-wide strategy to develop metacognition in gateway science courses. Paper presented at the POD Network Conference, Portland, Oregon.

Kaldor, E., & Swanson, H. (2019). How can you elevate metacognition on your campus? Try the Ace-Your-Course Challenge. The National Teaching & Learning Forum, 28(2), 5–7.

McGuire, S. Y. (2015). Teach students how to learn: Strategies you can incorporate into any course to improve student metacognition, study skills, and motivation. Sterling, VA: Stylus Publishing.

Nilson, L. B. (2013). Creating self-regulated learners: Strategies to strengthen students’ self-awareness and learning skills. Sterling, VA: Stylus Publishing.