Author: Lauren Scharff

The Promotion of Metacognition Through Soft Skills

by Mary Hebert, Fairleigh Dickinson University

 Downloadable

Description of Activity:

I teach a course in Metacognitive Strategies which focuses on the social and emotional components to academic success. These are referred to as ‘soft skills’ (emotional intelligence, interpersonal and intrapersonal awareness, emotional regulation, problem solving etc.) The course is presented to students who are members of the Regional Center at FDU who have been diagnosed with a language-based learning disability and or ADHD/ADD. Weekly journal reflections are completed based on a prompt that reflects a soft skill that is being addressed in the lecture. These journal entries serve as a means of enhancing metacognition and reflection of the material and focus on strategies of incorporating the skill into practice of academic performance.

An additional element involves a final project which requires the students to identify an individual they have admired for their successful accomplishment of some specific achievement. They are required to interview this individual and discover the soft skills associated with their accomplishment, not the ‘hard skills’ which are traditionally aligned with success (GPA, School Attended, Titles achieved etc). Furthermore, the student specifically is asked to assess their own soft skill set, including areas that are strengths and those to develop, and implement a plan of incorporating these into their academic goals and pursuits.

Further details of the activities can be found here.

Motivation and Context:

The class is designed to explore the ‘soft skills’, which include the social and emotional skills that are associated with academic success. The assignments are designed to provide tangible exercises that, when explored in a metacognitive manner and applied purposefully with a plan, can result in success and improve the academic and career course of an individual. The goal is improved self-regulation and critical thinking in regard to specific social and emotional skills that are highly correlated with academic success.

Nuts and Bolts:

The specific intent of this course and its assigned exercises is to weave an academic experience with the content of metacognition and soft skills that are connected to academic and career success. Specific topics addressed include emotional intelligence, personal responsibility, grit, self-motivation, interdependence, active listening, self-awareness, life-long learning, motivation, growth mindset, and goal setting. Students participate in discussion, reflection exercises, and the final project requiring them to take the knowledge of soft skills presented in class, think critically and analyze these topics, and implement them by carrying out an interview and create a presentation. The final project of interview and presentation is a culmination of analyzing a ‘story of success’ that from a distance may have looked easily attained for the interviewee. The task is for the student to discover through inquiry about soft skills, how in fact these played a critical role in the successful outcome for the interviewee. The students acquire insight into the ‘reality’ of the achievement, reflect on the soft skills they have developed and ones that they would benefit from developing further. A key feature is working on the plans of implementation which demonstrates improved critical thinking and capacity for self-regulation of good decision making and goal attainment.

The result is metacognitive ‘boot camp’ in regard to the less frequented content in the classroom that are key to academic effectiveness. Metacognition has been associated with improved critical thinking skills (Magno, 2010). Students are given knowledge about soft skills, asked to discuss through oral and written means of reflection, and then take it a step further and asked to apply the concepts to their own academic tasks throughout the semester. This sequence of knowledge acquisition, analysis, and application are the nuts and bolts of weaving the material together.

Outcomes:

The highly interactive nature of the course forces the contemplation necessary for students to adopt a more metacognitive approach to learning and their goals beyond the classroom. Critical thinking and self -regulation related to the connection between soft skill development and their academic and learning capacity is improved. As a counselor within the program that serves the students, I meet with each student individually one time per week during their freshmen year. I have observed that students begin to synthesize the course material with their academic functioning and improve their approach to matters related to their courses, studying, and academic goals. Many students begin to consider options to their approach in regard to their broader education and learning environment.

The culminating final project results in enhanced awareness of the interdependent nature of soft skills and hard skills for overall success in learning and career effectiveness. Presentations have been extraordinarily diverse with students choosing political figures, doctors, artists, students, business people, professors, peers, parents, coaches etc. Each year the series of presentations showcases the synthesis of soft skills and how growing awareness and purposeful use of these optimizes success academically as well as in career endeavors. Students demonstrate through their writing and oral reflection of their own use of soft skills, goals of further developing targeted soft skills during college to assist them in achieving academic success as well as future career success.

Lessons Learned and Future Directions:

The literature is clear in support of the importance of soft skills both in the classroom and in life. While some time during the course is spent connecting the material to career endeavors, future directions might include more of this element. In addition, it would be worthy to have a ‘maintenance program’ that extends beyond the time of the course, so that as the freshmen students progress, they are provided with opportunities to review and integrate the soft skill concepts throughout their remaining years of their college experience.

As a higher order thinking strategy, metacognition offers the opportunity to enhance and tap into the potential of the brain power within each student. Greater flexibility and awareness in thinking is the outcome and the continued goal of this form of application of metacognition.

Reference

Magno, C. Metacognition Learning (2010) 5: 137. doi:10.1007/s11409-010-9054-4


Metacognitive Reading Boosts Philosophy Exam Scores

by John Draeger, SUNY Buffalo State

 Downloadable

Motivations and context

I teach philosophy at a state university with approximately 10,000 undergraduates. I started incorporating the following metacognitive reading activity in order to promote the deep thinking and synthesis that students often struggle with on my essay exams. The bulk of my teaching falls within the general education curriculum where I help students develop help students develop critical skills (e.g.,close reading, careful writing, critical thinking) as well as expose them them to big concepts. I want students to see that topical topical debates over abortion, euthanasia, and hate speech often boil down to similar big conceptual issues (e.g., how to balance individual liberty against government intrusion, how to assess the benefits of individual expression against the harm to others). My exam questions typically ask students to consider the views of three authors across topical debates (e.g., one writing on abortion, one on euthanasia, one on hate speech) and then discuss which two authors are most alike and which are most different. Some students are stuck almost immediately because they have a hard time seeing how the conceptual issues could be at all alike when the topical issues are so different. These students resort to summarizing the authors. Some students can begin to see the underlying conceptual connections, but they often have difficulty developing those ideas. Both groups of students are left wondering how they could earn full credit on the exam.

Nuts and bolts

I’ve started asking students a series of questions that help make their thoughts about the writings and their own thinking about the writings more explicit, ultimately supporting their synthesis of the different authors and concepts for the exams. In order to prevent these questions from being interpreted as busy work, I introduce metacognition on the first day class. I explain that I want them to learn how to learn, and the writing assignments will help them figure out how to develop the type of thinking required for this course.

The questions fall into three categories. The first category alerts students to importance of having a reading strategy and being engaged.The second category pushes students beyond mere identification of an author’s thesis towards identifying the underlying issues. The third category prompts students to reflect on how the reading led to their identification of the underlying issues. This last category is the most metacognitively focused and important for helping them synthesize their understanding.

  1. General — what was the most challenging part of the reading? What was the most useful part? What was your reading strategy? How might you approach the reading differently next time?
  2. Conceptual issues — what was the central issue in the reading? How are the central conceptual issues related to the author’s thesis? How does this author frame the central issue compared to the other? How might this author respond to the previous author?
  3. Putting it altogether — what is a passage in the reading that illustrates the underlying issue? What is the evidence that the author takes this issue to be central? What is your strategy for uncovering these issues? How would you know if you’re correct? How would you change your approach if you’re not).

Because I want my students to be on a “steady diet” of metacognitive reflection, students are asked at least one question from each of the three categories as part of their preparation for each lesson. Responses to each question tend to be approximately a paragraph in length, and and they are graded pass/fail. Grades are determined less by the accuracy of the content, but by whether they made a “good faith” effort, which reduces the grading load.

Outcomes

Because students are required to explicitly practice with sort of thinking at the heart of the course, students are in a position to engage their own learning, which enables them to monitor their progress and make adjustments as necessary (e.g., ask questions in class, adapt reading strategies, attend office hours). When it comes time for the exam, students are better prepared for the type of thinking they are required to display and express much less confusion about what is being asked of them. As I grade the exams I am pleased to observe that many fewer of them resort to simply summarizing authors and they at least attempt to engage in the required type of thinking.

Lesson learned and future directions

Prior to this metacognitive activity, I thought that I was being clear about the type of thinking that I required of students, and I thought they were receiving plenty of opportunities to practice during class discussion. Even with this activity, however, I believe I need to provide students with more opportunities to become explicitly aware of their thinking and how to modify their strategies to achieve success.


Encouraging Metacognition in the Advanced Physics Lab

by Melissa Eblen-Zayas, Carleton College  Downloadable

 

Description of activity:

I have incorporated metacognitive support activities in the form of written reflections and class discussions to help students develop better approaches to dealing with challenges that arise in open-ended experimental work in an advanced lab course in physics.

Motivations and context:

The advanced lab course is the third of three required intermediate/advanced courses for the physics major that has a significant lab component. This course typically enrolls 18-24 physics majors, and the labs are significantly less scripted than the other required lab courses. The laboratory activities consist of three two-week-long, instructor-designed labs and four weeks of students carrying out an experimental project of their own design.

While some students welcome the move to more open-ended laboratory work, others struggle. Some students are reluctant to take initiative; rather than trying to problem solve on their own, they seek help from course instructors as soon as problems arise. Other students have difficulty developing a strategic approach to troubleshoot the challenges they encounter. To encourage independence in the lab, I have introduced reflection prompts to support student metacognition. Encouraging students to reflect on how they approach challenges and how they will do things differently going forward helps students develop more thoughtful problem-solving approaches in open-ended laboratory work, thereby increasing self-sufficiency and reducing frustration.

Nuts and bolts:

One of the four course goals for the advanced lab course is that students will demonstrate the ability to be reflective on the practice of experimental physics. I introduce the importance of reflective practice on the first day of course, and incorporate reflection activities in both the two-week instructor-designed labs and throughout the final project. These reflection activities account for 10% of the course grade, and most of these reflection activities are graded using a rubric.

1. First day of class. Prior to the first class, I ask students to respond to the prompt: “In two sentences, describe your definition of a successful experiment.” Then I select a number of student statements and share them the first day of class. Although student definitions of a successful experiment vary widely, many responses fall into one of two categories; a successful experiment is a) an experiment that gives a result that is in agreement with what is expected, or b) an experiment in which the experimenter learns something (maybe not what they intended). We discuss these two definitions of successful experiments, and I encourage students to adjust their expectations and appreciate that learning from things that go wrong is still a “success” in the experimental realm. These conversations allow me to introduce the importance of metacognition and the course goal of helping students become reflective practitioners.

2. Reflections on the instructor-designed labs. At the end of every two-week instructor-designed lab activity, I ask students to reflect on their most recent lab and respond to five questions designed to foster metacognition:

  1. Tell me a bit about how you approached the lab.
  2. When you ran into problems, what was the strategy your group employed for troubleshooting the problems you encountered?
  3. What types of pre-reading or additional research did you do to prepare for this lab?
  4. When you asked for help, who did you seek help from (other members of your group, other groups, your lab assistant, your instructor) and what kinds of questions did you ask?
  5. What is one thing that you will do differently when tackling labs going forward?

Students write individual responses to these prompts, and I provide feedback using the rubric. When I first began using these reflective prompts, I did not grade them. Grading the responses has increased the quality and depth of the reflections.

3. Reflections on the final project. I ask students to reflect on their final project work throughout the course of the project. Here is a sample of the questions used:

  1. What did you learn from the process of identifying and refining your final project proposal? What are you most looking forward to and what do you anticipate the biggest challenge will be as you begin working on your final project?
  2. What aspect of your contributions to the final project demonstrates your strengths and talents and why?
  3. What is one significant problem that your group encountered when working on your project in the past week, and how did you overcome it or redesign your project to work around it?
  4. What are your main project goals for the coming week, and how do you plan to pursue those goals?

The format for the responses has varied over the years. Sometimes lab groups respond to one of these prompts during a short oral report to the whole class. Other times, students write individual responses. Still other times, one of these questions serves as the starting point for an in-class discussion. I have found benefits and drawbacks to each of these approaches, and I continue to experiment with the format.

Outcomes:

Including metacognitive support activities in the advanced lab course, being explicit about why reflection is important in experimental physics, and grading student reflective responses has had a positive impact on the quality of student reflections and student attitudes towards the course. Students develop a more self-sufficient approach to tackling challenges that they encounter in the lab, and frustration is reduced. I reported some of the outcomes in a paper presented at the 2016 Physics Education Research Conference. That paper has been published in the conference proceedings:

Eblen-Zayas, M. (2016). The impact of metacognitive activities on student attitudes towards experimental physics, In D. L. Jones, L. Ding, & A. Traxler (Eds). 2016 PERC Proceedings, 104, doi:10.1119/perc.2016.pr.021


Addressing Metacognition Deficits in First Semester Calculus Students: Getting Students to Effectively Self-Evaluate their Understanding

by Derek Martinez, University of New Mexico

 Downloadable

Motivations and context: 

The problem I chose to tackle as a UNM Teaching Fellow was to develop methods for how to teach first semester calculus students to effectively self-test and develop metacognitive skills. One of the biggest issues I have seen over the years is students thinking they understand the material, getting over confident, and then performing horribly on an exam. The practices described below were carried out during the spring 2016 semester in two math 162 (Calculus I) classes.

Method: The two main ways that metacognitive strategies were incorporated into the curriculum were (1) daily “Test Yourself” exercises and (2) exam skills check self-assessments (essentially practice tests) before each exam. The “Test Yourself ” exercises were designed to be a daily reminder to the students that they should not confuse the ability to follow a lecture with the ability to solve a problem on their own. The purpose of the self-assessments was to help students identify where they had gaps in their understanding before taking each exam.

The “Test Yourself” exercises (example attached) were e-mailed to students the night before the lecture and were designed to give students a way to assess whether or not they understood the fundamental concepts of the lecture the following day. For example, if the lecture that day was about rates of change applications, the exercise would focus on an easy-to-medium level example that would test whether or not the students got the fundamental concepts from the section before going on and trying the more challenging homework problems.

Sending the exercises out the night before was effective in getting many students to read ahead in the text, and try to solve parts of the exercises (this was especially true of students who were struggling, or had math anxiety). If a student had solved the exercise before class, they were instructed to bring in a blank exercise and make sure they could duplicate their results without notes. The format for each class was usually lecture for about 40 minutes and then students would work on these exercises (some in groups, some by themselves).

The self-assessments (example attached) were given about five days before each exam. Participation in this was voluntary. I reserved a room outside of class and students took this like an actual exam. I made it clear to the students that the material on these assessments covered the fundamental ideas and basic examples, but were at a lower level of difficulty than the actual exams. The reasoning behind this was to help students pinpoint what core skills they still needed to work on. I graded these assessments just like exams so students could get feedback on their work as well as use of proper notation. To help identify their level of metacognition, at the end of each assessment the students were asked to rank their performance on a scale of 1-5 (5 being best performance). In many cases this ranking followed by actual exam scores provided further evidence to the students that they tended to be overconfident in their preparedness and needed to study more. In the beginning, students tended to over rank their performance but by the final exam assessment, their rankings were more in line with their performance.

Outcomes: Students in my spring 2016 sections had a final exam pass rate of more than 11% higher than all other sections (group graded without me to avoid any possible bias). These students also had a higher final exam pass rate than my fall/spring 2015 students by about 10% (when I did not yet incorporate these activities). The self-assessments seemed to have the biggest measurable impact on student success, as students who took them consistently outscored those who did not by 10 – 20% on the exams. Further, scores on the actual exams were 15 – 65% higher than on the self-assessments. I believe this was due to the fact that they guided and motivated their learning as well as simply scared some students into studying harder.

Lessons learned: “Buy-in” from the beginning is essential. Sharing the data with the students after the first assessment significantly increased the number of students taking the remaining assessments. These were mainly STEM majors so the statistical evidence went a long way with them. It was also crucial to make time throughout the semester to talk about what metacognition is and remind the students why they were doing these exercises.


Weekly Status Reports to Promote Awareness

by David Woods and Beth Dietz, Miami University

 Downloadable

Motivation for the activity or process: Teaching an introductory Information Technology (IT) course involves several goals that focus on creating metacognitive awareness and cognitive monitoring (Flavell, 1979; Schraw, 1998). The main goal of the course is to introduce students to several IT topics (e.g., data representations, computer architecture, and assembly language) that are foundational to the IT curriculum. Other goals of the course include analyzing and solving problems using a computer programming language, as well as applying written and oral communication skills to IT. Teaching these skills also helps address misconceptions about what IT professionals actually do. Students are often surprised to learn that IT professionals usually work in teams for specific projects or on an ongoing basis. Status reports are a key communication tool for groups, and good status reports require the individual to reflect and analyze what they have done, and plan for the future. Considering the course as a project, the status report should prompt the planning and evaluation aspects of metacognitive regulation (Flavell, 1979).

Context: A metacognitive-awareness activity was used in an introductory IT course. The course is a 100-level course and is one of the first courses taken by students considering a major in Computer and Information Technology. Typically, the class size is 20 – 25 students. While the instructor was only in his second year of full time teaching, he also had over 15 years experience working as an IT professional.

Description of activity: Weekly status reports are common activities in many IT positions, especially when an individual is part of a larger project team. They are a basic way for an employee to document what they have accomplished and what they are currently working on. This is valuable in the IT field since work such as writing software or configuring a server does not produce physical objects that provide visual evidence of progress.

The requirements for the status report were simple and made use of several metacognitive processes (Fogerty, 1994). Students were asked to discuss three specific items:

  • Current week activity: List the main course related activities since the last status report and provide a brief discussion of each along with the amount of time spent on the activity. This prompts the student to evaluate their learning from the past week.
  • Upcoming activity: List major course related activities planned for the next week with a brief discussion of the activity and what will be completed during the week. This prompts the student to plan the learning for the next week.
  • Issues and Overdue items: List any problems with the course materials or assignments. If there are no issues, this should be clearly stated. This prompts the student to monitor their understanding of the issues or problems.

During the semester, students completed 13 status reports. The status reports made up 5% of the final grade and students were allowed to skip three reports (or alternatively earn extra points by doing all of the assigned status reports).

Outcomes and Lessons Learned: The assignment met the immediate goal of prompting metacognitive reflection by asking students to evaluate their prior learning, plan for future learning, and monitor the learning process (Fogerty, 1994). In addition, the status reports gave the instructor good feedback on the amount of work that students did outside of the scheduled class meetings. An additional benefit was the opportunity to provide feedback to students who submitted status reports with limited content and limited evidence of planning and evaluation.

Many status reports showed clear evidence of evaluation and planning as students reported challenges with specific concepts or assignments and then planned activities in response. Some students failed to mention class meetings or submitted assignments in the current week activity. When this was mentioned in grading feedback, later status reports from these student showed improved tracking of completed work.

As the semester progressed and a few students missed assignments, there was an opportunity to ensure that these were noted and discussed in the overdue items section. In several instances, instructor comments led to students evaluating root causes including poor time management and mandatory overtime at work. Not all of the root causes had obvious solutions, but discussing the root causes offered a chance to plan ways to address the issue and was more productive than simply reminding students about late assignments.

The simple structure for the status reports should work well for courses at all levels. In courses where students have more than a week to complete assignments, status reporting could require students to break assignments down into smaller tasks, which is a useful skill to develop.

References:

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34 (10), 906-911.

Fogarty, R. (1994). How to teach for metacognition. Palatine, IL: IRI/Skylight Publishing.

Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(1-2),113-125.


Practice with a Reasoning Process to Make Learning Visible and Improve Academic Performance

by Jessica Santangelo, Hofstra University

 

Downloadable

Description of Activity

Motivations and context: I teach a fast-paced, content-heavy introductory biology course. Many students struggle in the course – not because they are not capable, but because they lack a repertoire of learning strategies that best support learning within the structure of the course. Rather than discuss “study strategies” as an add-on to course content, this activity has students model behaviors that make their learning visible, reduce reliance on memorization, and empowers them with a process to improve academic performance.

My basic goal with this activity was to make a very specific process available to students to mitigate the tendency I saw of students, when faced with a challenging question or concept, to simply guess or give up. Namely, students remember one key fact about a complex system from which all other pertinent facts can be derived. In this specific example, they organize those facts in a table and (critically) use the table when faced with questions regarding the system. The process of reasoning from a key fact to a deeper or more applied understanding is not metacognitive in and of itself. In this case, metacognitive development is promoted by the structuring of the in-class work that allows multiple opportunities for practice with the reasoning process.

Nuts and bolts of an example application

In the course we cover the urinary system. The most challenging aspect of this topic is the function of antidiuretic hormone (ADH). It involves understanding the effects of a diuretic (so students can then understand the effects of an antidiuretic) and osmosis – the movement of water across a semipermeable membrane. It further involves blood pressure, blood osmolarity, stimuli that either cause or inhibit release of ADH from the hypothalamus, and impacts of ADH (or lack thereof) on the kidney. Needless to say, there are a lot of moving parts.

I structure two class periods around one concept: Diuretics promote urine production. I tell students that this is the one thing they should memorize. Everything else follows from that one statement. So, rather than memorizing the entire table below, they memorize one statement, then reason their way through all the other information. Making students aware of this general strategy can greatly reduce the amount of time spent memorizing while increasing the amount of time spent making connections between interrelated facts or processes. Indeed, it’s worth asking students to self-identify one key starting point for any concept such that, if they remember that one key point, they can reason through the rest of the information.

The one concept to remember: Diuretics promote urine production.
Diuretic Antidiuretic
urine production Increases / promotes Decreases / inhibits
water loss Increases / promotes Decreases / inhibits
water retention Decreases Increases
blood osmolarity Increases (more salty as remove water) Decreases (less salty as add water)
blood pressure Decreases (as remove water) Increases (as add water)

I introduce the one concept, then have students work in groups to fill in the table on large wall-mounted whiteboards. Throughout their group work I ask questions to promote their metacognitive development like “What do you already know?” and “How did you come to that conclusion?”. This is a key step in the metacognitive process: asking them to make their reasoning visible to themselves and their group-mates. Though students may get stuck, being metacognitive (i.e., asking “what do I know, how do I know it, and how does that help me?”) helps them reason their way through more effectively. At the end of the class session I remind students to test themselves on their ability to start with the one key concept and subsequently explain the table before coming to the next class session.

The next class session, students put all of their notes and other resources away, and recreate the table on the wall-mounted whiteboards using only their brains. Invariably, most groups jump right into filling out the table. But one or two groups will take the time to write “Diuretics promote urine production” on their board before filling in the table. The groups who write this tend to complete the table more quickly and more accurately. I use this as a teachable moment for all the groups by reminding them that they have a simple tool – the one phrase to remember – to guide them in completing the table.

The groups then use their tables as a guide to answer a series of challenging questions about the stimuli for ADH release/inhibition and the associated outcomes. Most groups get bogged down in the questions – they discuss possible answers with their neighbors but go round and round and get confused. I let this happen for a question or two and then I remind students to use the table they put on the board. I ask one student from each group to stand up and model (with their group’s help) how to use the table to answer the next question. At this point, there are lots of “oh”s and “aha”s as students realize it is much easier to arrive at the correct answer using the table.

I then tell students: “You just used a tool (the table) to help you answer this question. What tools do you have available to you when you face a question like this on the exam?” Most of them look around in bewilderment as I don’t allow them to use any outside resources on exams. I then ask “What about the table?” and they say “But we aren’t allowed to bring anything with us to the exam”. And I say “But where did that table come from today?” and they respond “our brains” and I reply, “Exactly. You remembered ONE sentence and then you filled out that whole table with just your brain. So why not jot that table down on your exam?” And their eyes light up…

This is another key step in the metacognitive process: making it obvious to students how they can use this approach on their own to support learning and achievement. The behaviors they modeled in class (remembering one key concept from which to derive all other relevant information, organizing information into an easy-to-reference format, and utilizing that organized information to answer applied questions) should not be used solely in class or when I ask them to do it. They can use those behaviors on their own to promote learning outside of class or on an exam. I have found that unless I make this explicit to students, they rarely use an approach from the classroom on their own.

Outcomes

I’ve been incorporating a variety of activities and practices to promote student metacognitive development into the course for a few years with success. As a result, many students who would not have passed (or would have barely passed) the course have altered their learning strategies and improved their grades – some to A’s and B’s. As I’ve incorporated this specific example with the urinary system I’ve noticed that students are more willing to attempt the challenging ADH questions and are more likely to reason out the answer than to simply guess.

Lessons learned and future directions

Modeling behaviors in a group context works well for these students. Most of them were not challenged in high school the way they are challenged in this course. Embedding tips and tricks that enhance their ability to make their thought process visible (i.e., that promote metacognition) within the very context of the course 1. makes the tips/tricks an inherent part of learning biology rather than “add-ons” and 2. Increases the likelihood that they will use these metacognitive tips/tricks (self-regulation). My goal is to have students model these behaviors with more topics in the course, constantly reinforcing the thought/reasoning process so it is ingrained by the end of the semester.


The impact of metacognitive activities on student attitudes towards experimental physics

This article by Melissa Eblen-Zayas, Ph.D., shares the implementation of metacognitive activities in an advanced Physics lab. She reports that “the introduction of metacognitive activities in an advanced lab where the laboratory work is not carefully scripted may improve students’ enthusiasm for experimental work and confidence in their ability to be successful in such work.” Check out this article to see the metacognitive prompts they used as well as learn about other metacognition-related activities.

Eblen-Zayas, M. (2016). The impact of metacognitive activities on student attitudes towards experimental physics. 2016 PERC Proceedings edited by Jones, Ding, and Traxler; doi:10.1119/perc.2016.pr.021


Metacognitive Awareness of Learning Strategies in Undergraduates

This article by Jennifer McCabe presents the results of two studies focusing on metacognitive awareness of learning strategies in undergraduates. Participants were asked to evaluate and predict the outcomes of six educational scenarios describing the strategies of dual-coding, static-media presentations, low-interest extraneous details, testing, and spacing. Study 1 showed low awareness of all strategies except for generation; and a correlation of scenario prediction accuracy with an independent metacognition scale. Study 2 showed improved prediction accuracy for students who were specifically taught about these principles in college courses. “This research suggests that undergraduates are largely unaware of several specific strategies that could benefit memory for course information; further, training in applied learning and memory topics has the potential to improve metacognitive judgments in these domains.”

McCabe, J. (2011). Metacognitive awareness of learning strategies in undergraduates. Memory & Cognition, 39, 462–476. doi:10.3758/s13421-010-0035-2


Glimmer to Glow: Creating and Growing the Improve with Metacognition Site

by Lauren Scharff, Ph.D., U. S. Air Force Academy *

It’s been three years since Improve with Metacognition (IwM) went live, but the glimmer of the idea started more than a year prior to that, and we still consider it a work in progress. The adventure started with a presentation on metacognition that Aaron Richmond and I gave at the Southwestern Psychological Association (SWPA) convention in 2013. We both had independently been working on projects related to metacognition, and decided to co-present in the teaching track of the conference. We had good attendance at the session and an enthusiastic response from the audience. I made the suggestion of forming some sort of online community in order to continue the exchange of ideas, and passed around a sign-up sheet at the end of the session.

I have to say that my initial idea of an online community was very limited in scope: some sort of online discussion space with the capability to share documents. I thought it would be super quick to set up. Well, the reality was not quite so easy (lol) and our ambitions for the site grew as we discussed it further, but with help from some friends we got it going just in time to unveil it at the SWPA 2014 convention. Along the way I pulled in our third co-creator, John Draeger, who helped shape the site and presented with us at the 2014 convention.

As Aaron mentioned in his reflection last week, during the past three years we have shared information about the site at a variety of conferences both within the United States and beyond. The response has always been positive, even if not as many people go the next step and sign up for updates or write guest contributions as we’d like. One common line of questioning has been, “This is fantastic! I am interested in doing something similar on the topic of X. How did you get it going?”

We do hope that IwM can serve as a model for other collaboration sites, so here are a few things that stand out for me as I reflect on our ongoing efforts and the small glow we have going so far.

  • Partnerships are essential! John, Aaron, and I have some different skill sets and areas of expertise relevant to running the site, and our professional networks reach different groups. Further, with three of us running it, when life gets nuts for one of us, the others can pick up the slack. I can’t imagine trying to set up and maintain a site like IwM all on my own.
  • Practice metacognition! The three of us periodically join together in a Skype session to reflect on what seems to be working (or not), and share ideas for new features, collaboration projects, etc. We use that reflection to self-regulate our plans for the site (awareness plus self-regulation –> metacognition). Sometimes we’ve had to back off on our initiatives and try new strategies because the initial effort wasn’t working as we’d hoped. A long-time saying I’m fond of is, “the only way to coast is downhill.” Any endeavor, even if wildly successful at first, will require some sort of ongoing effort to keep it from coasting downhill.
  • Be open and provide an environment that supports professional development! (And realize this requires time and effort.) We want to encourage broad involvement in the site and provide opportunities for a wide variety of people interested in metacognition to share their ideas and efforts. We also hope to have a site that is viewed as being legitimate and professional. This balancing act has been most apparent with respect to the blog posts, because not everyone has strong writing skills. And, we believe that even those with strong writing skills can benefit from feedback. Thus, we provide feedback on every submitted post, sometimes suggesting only minor tweaks and sometimes suggesting more substantial revisions. The co-creators even review each other’s drafts before they are posted. As anyone who provides feedback on writing assignments or reviews journal articles knows, this process is a labor of love. We learn a lot from our bloggers – they share new ideas and perspectives that stimulate our own thinking. But, providing the appropriate level of feedback so as to clearly guide the revisions without squashing enthusiasm is sometimes a challenge. Almost always, at least two of the co-creators review each blog submission, and we explicitly communicate with each other prior to sending the feedback, sometimes combined and sometimes separate. That way we can provide a check on the tone and amount of feedback we send. Happily, we have received lots of thanks from our contributors and we don’t have any cases where a submission was withdrawn following receipt of our feedback.

Upon further reflection, my overall point is that maintaining a quality blog, resource, and collaboration site requires more than just getting people to submit pieces and posting articles and other resources. We hadn’t fully realized the level of effort required when we started, and we have many new ideas that we still hope to implement. But, on so many levels all the efforts have been worthwhile. We believe we have a fantastic (and growing) collection of blogs and resources, and we have had several successful collaboration projects (with more in the works).

We welcome your suggestions, and if you have the passion and time to help us glow even brighter, consider joining us as either a collaboration-consultant or as a guest blogger.

Lauren

* Disclaimer: The views expressed in this document are those of the authors and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


The Strategy Project

This study presents an instructional method that requires deliberate practice of self-regulated learning strategies including active reading, management of study time and achievement goals, proactive interaction with faculty, and metacognitive reflection within the context of a student-selected course. Four instructors implemented the assignment–called “The Strategy Project”–in their first-year seminar courses, and student reflection papers were analyzed for emerging themes. These themes suggest the positive impact of applying pedagogy that requires intentional within-course application of self-regulated learning strategies, suggesting the Strategy Project may be a viable way to teach and encourage college-level strategic behavior.
For more information about this study, follow the link below:

New Year Metacognition

by Lauren Scharff, Ph.D., United States Air Force Academy *

Happy New Year to you! This seasonal greeting has many positive connotations, including new beginnings, hope, fresh starts, etc. But, it’s also strongly associated with the making of new-year resolutions, and that’s where the link to metacognition becomes relevant.

As we state on the Improve with Metacognition home page, “Metacognition refers to an intentional focusing of attention on the development of a process, so that one becomes aware of one’s current state of accomplishment, along with the situational influences and strategy choices that are currently, or have previously, influenced accomplishment of that process. Through metacognition, one should become better able to accurately judge one’s progress and select strategies that will lead to success.”

Although this site typically focuses on teaching and learning processes, we can be metacognitive about any process / behavior in which we might engage. A new year’s resolution typically involves starting a new behavior that we might deem to be healthier for us, or stopping an already established behavior that we deem to be unhealthy for us. Either way, some effort is likely to be involved, because if it was going to be easy, we wouldn’t create a resolution to make the change.

Effort alone, however, is unlikely to lead to success. Just like students who “study harder” without being metacognitive about it, people who simply “try hard” to make a change will often be unsuccessful. This is because most behaviors, including learning, are complex. There are a multitude of situational factors and personal predispositions that interact to influence our success in obtaining our behavioral goals. Thus, it’s unlikely that a single strategy will work at all times. In fact, persisting with an ineffective strategy will lead to frustration, cynicism, and the giving up on one’s resolution.

Now, typically, I am not the sort of person who actually makes new-year resolutions. But this new year presents a new situation for me. I will be on sabbatical and working from home. I have prepared a fairly ambitious list of professional development activities that I hope to accomplish. I know I am capable of each of them. But, I also know that I will be working in an environment with a different group of distractions and without many external deadlines. Instead of committee work, grading, short turn-around taskers, and meetings with students and colleagues preventing me from working on my publications and other professional development activities, I will have a dog with big brown eyes who would love to go for a walk, children who need attention when they’re home from school, and projects at home that I usually can put out of mind when I’m at the office.

My resolution to myself for the coming 6 months of my sabbatical is that I will create a positive work environment for myself and accomplish my list of professional development activities while maintaining a balance with my family and personal goals. I know that I will need a variety of strategies, and that I will need to take time to reflect on the state of my progress and show self-regulation in my choice of strategies at different times. I plan to use a journal to help me with my awareness of the alignment between my daily goals and the activities in which I choose to engage in order to accomplish those goals.[1] This awareness will guide my self-regulation when, inevitably, I get off track. I also plan to make some public commitments and provide updates to my friends and colleagues regarding specific goals I plan to accomplish at specific times, as public commitment provides motivation, often results in social support, and is another way to encourage self-awareness and self-regulation, i.e. metacognition.

I’ll let you know how it goes in 6 months. 🙂  Meanwhile, Happy New Year and all the best to you with your new-year resolutions. Try using the tools of metacognition to help you succeed!

[1] See our preliminary research summary about the effectiveness of instructors using journals to enhance their metacognitive instruction.

* Disclaimer: The views expressed in this document are those of the author and do not reflect the official policy or position of the U. S. Air Force, Department of Defense, or the U. S. Govt.


The GAMES Survey: A Tool to Scaffold Metacognitive Practices

by Lauren Scharff, U. S. Air Force Academy

As many of us educators know, an unfortunately large number of students, both at the K-12 and college-levels, do not give much thought to how and why they try to learn the way they do, much less demonstrate strong habits of metacognition. Talking in general about metacognition might garner some students’ interest, but without some concrete guidance on how to engage in behaviors that support metacognition, students are less likely to develop such practices.

Thus, I was pleased to rediscover the GAMES survey / self-assessment tool created by Marilla Svinicki when I was re-reading her excellent book, Learning and Motivation in the Postsecondary Classroom, as part of a book group at my institution. GAMES stands for:

  • Goal-oriented studying
  • Active studying
  • Meaningful and memorable studying
  • Explain to understand
  • Self-monitor

For each component of the survey, there are five to ten behaviors for which students indicate their likelihood to perform using a 5-point scale ranging from “Never” to “Always.” These behaviors are distinct, tangible actions such as:

  • Analyze what I have to do before beginning to study. (Goal-oriented studying)
  • Ask myself questions before, during, and after studying. (Active studying)
  • Make connections between what I am studying and past classes or units. (Meaningful and memorable studying)
  • Discuss the course content with anyone willing to listen. (Explain to understand)
  • Keep track of things I don’t understand and note when they finally become clear and what made that happen. (Self-monitor)

Marilla suggests that the use of such an instrument can help students become more aware of the possibility of self-regulating their learning behaviors. This combination of awareness and self-regulation is key to metacognition, and is what is prompting this blog post.

Through the process of completing the GAMES survey, students are introduced to more than 30 specific behaviors that holistically will support metacognition about learning. Students can easily observe areas where they might show stronger or weaker engagement, and they can focus their efforts where they are weaker, using the list of specific, tangible behaviors as a scaffold to help them target their activity.

At my institution, the U. S. Air Force Academy, we plan to use the GAMES survey in a current Science of Learning workshop series for our students led by students. Most of the seminar attendees are students who are struggling academically, but we are advertising that, by “studying smarter, not only harder” students of all levels of academic achievement can improve their learning. We believe that the GAMES survey will help students target specific behaviors that have been shown to support deeper learning.

We are not the only institution that has seen value in disseminating the GAMES survey to students. For example, several years ago, Georgia Tech encouraged its use across all sections of their first-year seminar. Importantly, they didn’t simply ask students to complete the survey and that was it. They encouraged instructors to help students use the results in a meaningful way, such as by picking a weak behavior and striving to improve it over a 2-week time period, or by having students journal about changes they made and how those changes seemed to impact their academic performance.

This survey tool is appropriate across the disciplines and only takes a few minutes for students to complete. Its use and a short follow-on activity to encourage meaningful application would not add great burden to a faculty member or take much time from normal course activities. But, the pay-off could be large for individual students, both in that course as well as others if they transfer the principles into new contexts. It’s definitely worth a try!

——————

Svinicki, M. D. (2004). Learning and motivation in the postsecondary classroom. Bolton, MA: Anker Publishing Co.

If you do not have access to Marilla Svinicki’s book, you can read a short online overview of GAMES on the Association for Psychological Science website (2006), and obtain a pdf copy of the survey online.


The Challenge of Deep Learning in the Age of LearnSmart Course Systems (Part 2)

A few months ago, I shared Part 1 of this post. In it I presented the claim that, “If there are ways for students to spend less time per course and still “be successful,” they will find the ways to do so. Unfortunately, their efficient choices may short-change their long-term, deep learning.” I linked this claim to some challenges that I foresaw with respect to two aspects of the online text chosen for the core course I was teaching: 1) the pre-highlighted LearnSmart text, and 2) the metacognition-focused LearnSmart quizzing feature. This feature required students to not only answer the quiz question, but also report their confidence in the correctness of that response. (See Part 1 for details to explain my concerns. Several other posts on this site also discuss confidence ratings as a metacognition tool. See references below.) My stated plan was to “regularly check in with the students, have class discussions aimed at bringing their choices about their learning behaviors into their conscious awareness, and positively reinforcing their positive self-regulation of deep-learning behaviors.” 

This post, Part 2, will share my reflections on how things turned out, along with a summary of some feedback from my students.

With respect to my actions, I did the following in order to increase student awareness of their learning choices and the impact of those choices. Twice early in the semester I took class time to explicitly discuss the possible learning shortcuts students might be tempted to take when reading the chapters (e.g. only reading the highlighted text) and when completing the LearnSmart pre-class quizzes (see Part 1 for details). I shared some alternate completion options that would likely enhance their learning and long-term retention of the material (e.g. reading the full text without highlights and using the online annotation features). Additionally, I took time to share other general learning / studying strategies that have been shown through research to support better learning. These ways of learning were repeatedly reinforced throughout the semester (and linked to content material when applicable, such as when we discussed human learning and memory).

Did these efforts impact student behaviors and choices of learning strategies? Although I cannot directly answer that question, I can share some insights based on some LearnSmart data, course performance, and reflections shared by the students.

With respect to the LearnSmart application that quizzed students at the end of each chapter, one type of data I was able to retrieve was the overall percent of time that student LearnSmart quiz question responses fell into the following correctness and confidence categories (a metacognition-related evaluation):

  1. Students answered correctly and indicated confidence that they would answer correctly
  2. Students answered correctly but indicated that they were not confident of the correctness of their response
  3. Students answered incorrectly and knew they didn’t know the answer
  4. Students answered incorrectly but reported confidence in giving the correct answer

I examined how the percentage of time student responses fell in each category correlated with two course performance measures (final exam grade and overall course grades). Category 1 (correct and confident) and Category 3 (incorrect and knew it) both showed essentially a zero relationship with performance. There was a small positive relationship between being correct but not certain (Category 2). Category 2 responses might prompt more attention to the topic and ultimately lead to better learning. The strongest correlations (negative direction) occurred for Category 4, which was the category about which I was most concerned with respect to student learning and metacognition. There are two reasons students might have responses in that category. They could be prioritizing time efficiency over learning because they were intentionally always indicating they were confident (so that if they got lucky and answered correctly, the question would count toward the required number that they had to answer both correctly and with confidence; if they indicated low confidence, then the question would not count toward the required number they had to complete for the chapter). Alternately, Category 4 responses could be due to students being erroneous with respect their own state of understanding, suggesting poor metacognitive awareness and a likelihood to perform poorly on exams despite “studying hard.” Although there was no way for me to determine which of these two causes were underlying the student responses in this category, the negative relationship clearly indicated that those who had more such responses performed worse on the comprehensive final exam and in the course at large.

I also asked my students to share some verbal and written reflections regarding their choices of learning behaviors. These reflections didn’t explicitly address their reasons for indicating high or low confidence for the pre-class quizzes. However, they did address their choices with respect to reading only the highlighted versus the full chapter text. Despite the conversations at the beginning of the semester stressing that exam material included the full text and that their learning would be more complete if they read the full text, almost half the class reported only reading the highlighted text (or shifting from full to highlighted). These students indicated that their choice was primarily due to perceived time constraints and the fact that the pre-class LearnSmart quizzes focused on the highlighted material so students could be successful on the pre-class assignment without reading the full text. More positively, a couple students did shift to reading the full text because they saw the negative impact of only reading the highlighted text on their exam grades. Beyond the LearnSmart behaviors, several students reported increasing use (even in other courses) of the general learning / study strategies we discussed in class (e.g. working with a partner to discuss and quiz each other on the material), and some of them even shard these strategies with friends!

So, what are my take-aways?

Although this should surprise no one who has studied operant conditioning, the biggest take-away for me is that for almost half my students the immediate reinforcement of being able to more quickly complete the pre-class LearnSmart quiz was the most powerful driver of their behavior, despite explicit in-class discussion and their own acknowledgement that it hurt their performance on the later exams. When asked what they might do differently if they could redo the semester, several of these students indicated that they would tell themselves to read the full text. But, I have to wonder if this level of awareness would actually drive their self-regulatory behaviors due the unavoidable perceptions of time constraints and the immediate reinforcement of “good” performance on the pre-class LearnSmart quizzes. Unfortunately, at this point, instructors do not have control over the questions asked in the LearnSmart quizzes, so that particular (unwanted) reinforcement factor is unavoidable if you use those quizzes. A second take-away is that explicit discussion of high-efficacy learning strategies can lead to their adoption. These strategies were relatively independent from the LearnSmart quiz requirement for the course, so there was no conflict with those behaviors. Although the reinforcement was less immediate, students reported positive results from using the strategies, which motivated them to keep using them and to share them with friends. Personally, I believe that the multiple times that we discussed these general learning strategies also helped because they increased student awareness of them and their efficacy (awareness being an important first step in metacognition).

————

Some prior blog posts related to Confidence Ratings and Metacognition

Effects of Strategy Training and Incentives on Students’ Performance, Confidence, and Calibration, by Aaron Richmond October 2014

Quantifying Metacognition — Some Numeracy behind Self-Assessment Measures, by Ed Nuhfer, January 2016

The Importance of Teaching Effective Self-Assessment, by Stephen Chew, Feb 2016

Unskilled and Unaware: A Metacognitive Bias, by John R. Schumacher, Eevin Akers, and Roman Taraban, April 2016


Using Metacognition to select and apply appropriate teaching strategies

by John Draeger (SUNY Buffalo State) & Lauren Scharff (U. S. Air Force Academy)

Metacognition was a recurring theme at the recent Speaking SoTL (Scholarship of Teaching and Learning) conference at Highpoint university. Invited speaker Saundra McGuire, for one, argued that metacognition is the key to teaching students how to learn. Stacy Lipowski, for another, argued for the importance of metacognitive self-monitoring through the regular testing of students. We argued for the importance of metacognitive instruction (i.e. the use of reflective awareness and self-regulation to make intentional and timely adjustments to teaching a specific  individual or group of students) as a tool for selecting and implementing teaching strategies. This post will share a synopsis of our presentation from the conference.

We started with the assumption that many instructors would like to make use of evidence-based strategies to improve student learning, but they are often faced with the challenge of how to decide among the many available options. We suggested that metacognitive instruction provides a solution. Building blocks for metacognitive instruction include 1) consideration of student characteristics, context, and learning goals, 2) consideration of instructional strategies and how those align with the student characteristics, context, and learning goals, and 3) ongoing feedback, adjustment and refinement as the course progresses (Scharff & Draeger, 2015).

Suppose, for example, that you’re teaching a lower-level core course in your discipline with approximately 35 students where the course goals include the 1) acquisition of broad content and 2) application of this content to new contexts (e.g., current events, personal situations, other course content areas). Students enrolled in the course typically have a variety of backgrounds and ability levels. Moreover, they don’t always see the relevance of the course and they are not always motivated to complete assignments. As many of us know, these core courses are both a staple of undergraduate education and a challenge to teach.

Scholarly teachers (Richlin, 2001) consult the literature to find tools for addressing the challenges just described. Because of the recent growth of SoTL work, they will find many instructional choices to choose from. Let’s consider four choices. First, Just-in-Time teaching strategies ask students to engage course material prior to class and relay those responses to their instructor (e.g., select problem sets or focused writing). Instructors then use student responses to tailor the lesson for the day (Novak, Patterson, & Gavrin, 1999; Simkins & Maier, 2004; Scharff, Rolf, Novotny, & Lee, 2013). In courses where Just-in-Time teaching strategies are used, students are more likely to read before class and take ownership over their own learning. Second, Team-Based Learning (TBL) strategies also engage students in some pre-class preparation, and then during class, students engage in active learning through a specific sequence of individual work, group work, and immediate feedback to close the learning loop (Michaelsen & Sweet, 2011). TBL has been shown to shift course goals from knowing to applying and create a more balanced responsibility for learning between faculty and students (with students taking on more responsibility). Third, concept maps provide visual representations of important connections (often hierarchical connections) between important concepts. They can help students visualize connections between important course concepts (Davies, 2010), but they require some prior understanding of the concepts being mapped. Fourth, mind mapping also leads to visual representations of related concepts, but the process is more free-form and creative, and often requires less prior knowledge. It encourages exploration of relationships and is more similar to brainstorming.

Any of these three tools might be good instructional choices for the course described above. But how is an instructor supposed to choose?

Drawing inspiration from Tanner (2012) who shared questions to prompt metacognitive learning strategies for students, we recommend that instructors ask themselves a series of questions aligned with each of our proposed building blocks to prompt their own metacognitive awareness and self-regulation (Scharff & Draeger, 2015). For example, instructors should consider the type of learning (both content and skills) they hope their students will achieve for a given course, as well as their own level of level of preparedness and time / resources available for incorporating that particular type of teaching strategy.

In the course described above, any of the four instructional strategies might help with the broad acquisition of content, and depending upon how they are implemented, some of them might promote student application of the material to new contexts. For example, while concept maps can facilitate meaningful learning their often hierarchical structure may not allow for the flexibility associated with making connections to personal context and current events. In contrast, the flexibility of mind-mapping might serve well to promote generation of examples for application, but it would be less ideal to support content acquisition. Team-Based-Learning can promote active learning and facilitate the application of knowledge to personal contexts and current events, but it requires the instructor to have high familiarity with the course and the ability to be very flexible during class as students are given greater responsibility (which may be problematic with lower-level students who are not motivated to be in the course).   Just-in-Time-Teaching can promote both content acquisition and application if both are addressed in the pre-class questions. During class, the instructor should show some flexibility by tailoring the lesson to best reach students based on their responses to the pre-class questions, but overall, the lesson is much more traditional in its organization and expectations for student engagement than with TBL. Under these circumstances, it might be that Just-in-Time strategies offer the best prospect for teaching broad content to students with varying backgrounds and ability levels.

While the mindful choice of instructional strategies is important, we believe that instructors should also remain mindful in-the-moment as they implement strategies. Questions they might ask themselves include:

  • What are you doing to “check in” with your learners to ensure progress towards daily and weekly course objectives?
  • What are signs of success (or not) of the use of the strategy?
  • How can you  adjust the technique to better meet your student needs?
  • Are your students motivated and confident, or are they bored or overwhelmed and frustrated? Are your students being given enough time to practice new skills?
  • If learning is not where it needs to be or student affect is not supportive of learning, what are alternate strategies?
  • Are you prepared to shift to them? If not, then why not?

These prompts can help instructors adjust and refine their implementation of the chosen instructional strategy in a timely manner.

If, for example, Just-in-Time assignments reveal that students are understanding core concepts but having difficulty applying them, then the instructor could tweak Just-in-time assignments by more explicitly requiring application examples. These could then be discussed in class. Alternatively, the instructor might keep the Just-in-Time questions focused on content, but start to use mind mapping during class in order to promote a variety of examples of application.  In either case, it is essential that instructors are explicitly and intentionally considering whether the instructor choice is working as part of an ongoing cycle of awareness and self-regulation. Moreover, we believe that as instructors cultivate their ability to engage in metacognitive instruction, they will be better prepared to make in-the-moment adjustments during their lessons because they will be more “tuned-in” to the needs of individual learners and they will be more aware of available teaching strategies.

While not a magic bullet, we believe that metacognitive instruction can help instructors decide which instructional strategy best fits a particular pedagogical situation and it can help instructors adjust and refine those techniques as the need arises.

References

Davies, M. (2011). Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? Higher education, 62(3), 279-301.

Michaelsen, L. K., & Sweet, M. (2011). Team‐based learning. New directions for teaching and learning,(128), 41-51.

Novak, G., Patterson, E., Gavrin, A., & Christian, W. (1999). Just-in-time teaching:

Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall.

Richlin, L. (2001). Scholarly teaching and the scholarship of teaching. New directions for teaching and learning, 2001(86), 57-68.

Scharff, L. and Draeger, J. (2015). “Thinking about metacognitive instruction” National Teaching and Learning Forum 24 (5), 4-6.

Scharff, L., Rolf, J. Novotny, S. and Lee, R. (2011). “Factors impacting completion of pre-class assignments (JiTT) in Physics, Math, and Behavioral Sciences.” In C. Rust (ed.), Improving Student Learning: Improving Student Learning Global Theories and Local Practices: Institutional, Disciplinary and Cultural Variations. Oxford Brookes University, UK.

Simkins, S. & Maier, M. (2009). Just-in-time teaching: Across the disciplines, across the

academy. Stylus Publishing, LLC.

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11(2), 113-120.


The Challenge of Deep Learning in the Age of LearnSmart Course Systems

by Lauren Scharff, Ph.D. (U. S. Air Force Academy)

One of my close friends and colleague can reliably be counted on to point out that students are rational decision makers. There is only so much time in their days and they have full schedules. If there are ways for students to spend less time per course and still “be successful,” they will find the ways to do so. Unfortunately, their efficient choices may short-change their long-term, deep learning.

This tension between efficiency and deep learning was again brought to my attention when I learned about the “LearnSmart” (LS) text application that automatically comes with the e-text chosen by my department for the core course I’m teaching this semester. As a plus, the publisher has incorporated learning science (metacognitive prompts and spacing of review material) into the design of LearnSmart. Less positive, some aspects of the LearnSmart design seem to lead many students to choose efficiency over deep learning.

In a nutshell, the current LS design prompts learning shortcuts in several ways. Pre-highlighted text discourages reading from non-highlighted material, and the fact that the LS quiz questions primarily come from highlighted material reinforces those selective reading tendencies. A less conspicuous learning trap results from the design of the LS quiz credit algorithm that incorporates the metacognitive prompts. The metacognition prompts not only take a bit of extra time to answer, but students only get credit for completing questions for which they indicate good understanding of the question material. If they indicate questionable understanding, even if they ultimately answer correctly, that question does not count toward the required number of pre-class reading check questions. [If you’d like more details about the LS quiz process design, please see the text at the bottom of this post.]

Last semester, the fact that many of our students were choosing efficiency over deep learning became apparent when the first exam was graded. Despite very high completion of the LS pre-class reading quizzes and lively class discussions, exam grades on average were more than a letter grade lower than previous semesters.

The bottom line is, just like teaching tools, learning tools are only effective if they are used in ways that align with objectives. As instructors, our objectives typically are student learning (hopefully deep learning in most cases). Students’ objectives might seem to be correlated with learning (e.g. grades) or not (e.g. what is the fastest way to complete this assignment?). If we instructors design our courses or choose activities that allow students to efficiently (quickly) complete them while also obtaining good grades, then we are inadvertently supporting short-cuts to real learning.

So, how do we tackle our efficiency-shortcut challenge as we go into this new semester? There is a tool that the publisher offers to help us track student responses by levels of self-reported understanding and correctness. We can see if any students are showing the majority of their responses in the “I know it” category. If many of those are also incorrect, it’s likely that they are prioritizing short-term efficiency over long-term learning and we can talk to them one-on-one about their choices. That’s helpful, but it’s reactionary.

The real question is, How do we get students to consciously prioritize their long-term learning over short-term efficiency? For that, I suggest additional explicit discussion and another layer of metacognition. I plan to regularly check in with the students, have class discussions aimed at bringing their choices about their learning behaviors into their conscious awareness, and positively reinforcing their positive self-regulation of deep-learning behaviors.

I’ll let you know how it goes.

——————————————–

Here is some additional background on the e-text and the complimentary LearnSmart (LS) text .

There are two ways to access the text. One way is an electronic version of the printed text, including nice annotation capabilities for students who want to underline, highlight or take notes. It’s essentially an electronic version of a printed text. The second way to access the text is through the LS chapters. As mentioned above, when the students open these chapters, they will find that some of the text has already been highlighted for them!

As they read through the LS chapters, students are periodically prompted with some LS quiz questions (primarily from highlighted material). These questions are where some of the learning science comes in. Students are given a question about the material. But, rather than being given the multiple choice response options right away, they are first given a metacognitive prompt. They are asked how confident they are that they know the answer to the question without seeing the response options. They can choose “I know it,” “Think so,” “Unsure,” or “No idea.” Once they answer about their “awareness” of their understanding, then they are given the response options and they try to correctly answer the question.

This next point is key: it turns out that in order to get credit for question completion in LS, students must do BOTH of the following: 1) choose “I know it” when indicating understanding, and 2) answer the question correctly. If students indicate any other level of understanding, or if they answer incorrectly, LS will give them more questions on that topic, and the effort for that question won’t count towards completion of the required number of questions for the pre-class activity.

And there’s the rub. Efficient students quickly learn that they can complete the pre-class reading quiz activity much more quickly if they chose “I know it” to all the metacognitive understanding probes prior to each question. If they guess at the subsequent question answer and get it correct, it counts toward their completion of the activity and they move on. If they answer incorrectly, LS would give them another question from that topic, but they weren’t any worse off with respect to time and effort than if they had indicated that they weren’t sure of the answer.

If students actually take the time to take advantage of rather than shortcut the LS quiz features (there are additional ones I haven’t mentioned here), their deep learning should be enhanced. However, unless they come to value deep learning over efficiency and short-term grades (e.g. quiz completion), then there is no benefit to the technology. In fact it might further undermine their learning through a false sense of understanding.


Lean Forward, but Do It Metacognitively!

by Lauren Scharff, Ph.D. (U. S. Air Force Academy)

As the Director for the Scholarship of Teaching and Learning (SoTL) at my institution, a large part of my job description involves helping faculty intentionally explore new approaches and how they impact student learning. In other words – I work with forward-leaning faculty who are ready to try new things. So, I think a lot about how, when, and why faculty members adopt new pedagogies, tools, and activities, and about when, for whom, and in what contexts these new approaches enhance learning. This work dovetails nicely with the development and goals of metacognitive instruction.

As a reminder if you’re relatively new to our site, one of the premises we’ve previously shared here (e.g. Scharff, March 2015) and elsewhere (Scharff and Draeger, NTLF, 2015) is that Metacognitive Instruction involves the intentional and ongoing interaction between awareness and self-regulation, specifically with respect to the pedagogical choices instructors make as they design their lessons and then as they carry them out.

I was happy to see these connections reinforced last month at our 7th Annual SoTL Forum. Dr. Bridget Arend was invited to give a morning workshop and the keynote address. Along with James R. Davis, she is co-author of Facilitating Seven Ways of Learning: A Resource for More Purposeful, Effective and Enjoyable College Teaching. In her workshop Bridget dug into how to facilitate critical thinking, promote problem-solving, and support the building of skills (3 of the 7 ways of learning), while in her keynote she focused more strongly on the concept of matching student learning goals with the most effective teaching methods. She went beyond the usual discussion of tips and techniques to explore the underlying purpose, rationale, and best use of these [pedagogical] methods.

Dr. Bridget Arend giving the keynote address at the 7th Annual SoTL Forum at the U. S. Air Force Academy
Dr. Bridget Arend giving the keynote address at the 7th Annual SoTL Forum at the U. S. Air Force Academy

7_Ways_of_Learning
Books such as these can help support metacognitive instruction.

While Bridget did not explicitly use the term “metacognitive instruction,” it struck me that her message of purposeful choice of methods directly supported key aspects of metacognitive instruction, especially those related to awareness of our pedagogical decisions. We (instructors) should not incorporate pedagogies (or new tools or activities) just because they are the ones typically used by our colleagues, or because they are what was “done to us as students and it worked for us,” or because they are the “new, latest-greatest thing” we’ve heard about. Rather, we should carefully review our learning goals and consider how each possible approach might support those goals for our students and our context.

We should also be mindful of other factors that might influence our adoption of new approaches. For example, administrators or institutions often reward faculty who are leading the adoption of new technologies. Sometimes the message seems “the more new technologies incorporated the better” or “out with the old and in with the new” so a program or institution can market itself as being the most cutting edge in education. However, while many of us appreciate being rewarded or showcased for new efforts, we also need to pause to consider whether or not we’re really supporting student learning as well as we could with these practices.

Questions we should ask ourselves before implementation include, How will our new pedagogical approach or a new app really align with the learning goals we have for our students? Will all of our choices complement each other, or might they work at cross-purposes with each other? Realistically, there are a limited number of learning outcomes we can successfully accomplish within a lesson or even across a semester.

As we implement these new approaches and tools, we should ask additional questions. How are they actually impacting aspects of student engagement, attitudes towards learning, and ultimately, the learning itself? How might they be adjusted (either “in the moment” or in future lessons) as we use them in order to better support our learning goals for our students in our context? No group of students is the same, and the context also shifts over time. What worked well in the past might need adjusting or more radically changing in the future.

In sum, we know that no single approach is going to work for all learning goals or all students across all situations. But if we build our awareness of possibilities using resources such as Facilitating Seven Ways of Learning (and many other published papers and texts) to help guide our pedagogical choices; if we carefully attend to how our approaches affect students and student learning; and we if modify our approach based on those observations (and maybe using systematic data if we’re conducting a SoTL research project), then we WILL be more likely to enhance student learning (and our own development as metacognitive instructors).

Thus, lean forward as instructors, but do it metacognitively!

————————-

Davis, James R. & Arend, B. (2013). Facilitating Seven Ways of Learning: A Resource for More Purposeful, Effective and Enjoyable College Teaching. Stylus Publishing, Sterling, VA.

Scharff, L. & Draeger, J. (September, 2015). Thinking about metacognitive instruction. The National Teaching and Learning Forum, 24(5), p. 4-6. http://onlinelibrary.wiley.com/doi/10.1002/ntlf.2015.24.issue-5/issuetoc


Using Metacognition to Make International Connections

by Lauren Scharff, PhD, U. S. Air Force Academy and John Draeger, PhD, SUNY Buffalo State

If you’re one of our longer-term followers, you’ll notice that this post is a bit different from others on our site. We just wrapped up a fantastic week in Melbourne, Australia working with six colleagues from around the globe, and we want to share some of our metacognition endeavors and reflections with you. This experience was part of the second International Collaborative Writing Groups  (ICWG) that is an affiliate effort for the International Society for the Scholarship of Teaching and Learning (ISSoTL).

Eight groups were part of the ICWG. The groups formed in May and met virtually over the summer to focus their topics and develop an outline prior to the face-to-face meeting this past week. Our group’s topic was The Student Learning Process, and we focused our efforts on how metacognition would support the transfer of learning from one situation or context to another. We believe the transfer of learning is one of the ultimate goals of education because it supports lifelong learning and employability.

The group’s work on how metacognition supports the transfer of learning will be revealed when it’s published, but meanwhile, we will share some ways that metacognition was part of our experience of facilitating the group. We’ll start with some pictures to set the tone. The first shows our group working: from left to right, Lauren, Susan Smith (Leeds Beckett University, UK), Lucie S Dvorakova (Honors Student, University of Queensland, Australia), Marion Tower (University of Queensland), Dominic Verpoorten (IFRES-University of Liège, Belgium), Marie Devlin (Newcastle University, UK), and Jason M. Lodge (University of Melbourne, Australia), [John Draeger taking the pic]. The second gives you a sense of the overall setting, showing multiple groups all kept to task by savvy ICWG coordinators, Mick Healy (University of Gloucestershire, retired) and Kelly Matthews (University of Queensland). Fortunately, Mick and Kelly also built in some social time for community building. The third picture shows our group at the Victoria State Library, left to right: Dominique, Sam, Marion, Sue, Marion, John, Lauren and Jason.

ICWG_SLP_Working

ICWG_mult_groups

ICWG_SLP_Social

How Metacognition Found Its Way into Our Facilitating Experiences

If you read the home page of this site, you’ll notice that we loosely define metacognition as the intertwined awareness and self-regulation of a process/skill, specifically with the goal of developing that process or skill. Although the site is focused on metacognition as it relates to teaching and learning, it can refer to any skill or process. Facilitating a group can be much like teaching, but it involves some additional processes that might more traditionally be linked to leadership and communication.

We noticed ourselves using metacognition in the following aspects of our work:

Use of Language: Given the international character of the group, self-monitoring and self-regulation allowed us to navigate differences in language and underlying assumptions. For example, through our discussions we learned that academic faculty might be referred to as ‘staff,’ ‘tutor,’ ‘instructor’ or ‘professor.’ Individual courses might be referred to as ‘classes,’ ‘modules’ or ‘units’ of study.

Assumptions about education: Our discussion revealed differences in the structures of the university systems in different countries. When discussing how students might use their learning in one course to inform their learning in another, the two North Americans on the team (John and Lauren) tended to think about transfer learning between a diverse set of courses across a broad liberal arts core curriculum in addition to transfer across more closely related courses within a major. Because undergraduate education in Australia and the United Kingdom tend not to be structured around a broad core curriculum, members of the team from these countries tended to focus on transfer learning within a particular field of study.

As we drafted our text and created a survey that was to be used in four different countries, we each engaged in self-monitoring of the terms as the conversation was in progress and would regulate behavior accordingly. For example, someone would start by saying “I think that staff might…” but then quickly add “or perhaps you might say ‘professors.’” Similarly, we would use our newly developed awareness of the different educational structures to guide our discussion about how transfer of learning might be supported across all of our learning environments.

Management of Project Scope: Both transfer of learning and metacognition are vast areas of study. Given the wide variety of experiences and individual interests in our group, we explored a wide array of possible directions for our paper, some of which we decided we would table for follow-on papers (e.g. how student level of intellectual development might impact transfer of learning and the creation of a “toolkit” for instructors that would help them support transfer of learning). Moving the conversation in fruitful directions required that all of us remain mindful of the task at hand (i.e. working towards a 6000-word article). Self-monitoring allowed us to detect when an interesting discussion had gone beyond the scope of our current article and self-regulation more quickly brought us back to the task at hand.

In summary, the international character of the writing group added a depth and richness to the conversation, but it also increased the likelihood of misunderstanding and the challenge of group management. Self-monitoring and self-regulation allowed us to overcome those challenges.

Many thanks to our group members for a fantastic face-to-face experience, and we look forward to our continued exchanges as we finalize the paper and carry on with the follow-on papers.


The relationship between goals, metacognition, and academic success

In this article Savia Countinho investigates the relationship between mastery goals, performance goals, metacognition (using the Metacognitive Awareness Inventory), and academic success.

Countinho, S. (2007). The relationship between goals, metacognition, and academic success. Educate. 7(1), p. 39-47


Metacognitive Development in Professional Educators

Stewart, Cooper and Moulding investigate adult metacognition development, specifically comparing pre-service teachers and practicing teachers. They used the Metacognitive Awareness Inventory and found that metacognition improves significantly with age and with years of teaching experience, but not with gender or level of teaching (Pre-K though post-secondary ed levels).

Stewart, P. W., Cooper. S. S., & Moulding, L. R. (2007). Metacognitive development in professional educators. The Researcher, 21(1), 32-40.


Breaking the Content Mold: The Challenge of Shaping Student Metacognitive Development

by Dr. Lauren Scharff, U. S. Air Force Academy

We all know that it’s difficult to break long-term patterns of behavior, even when we’re genuinely motivated and well intentioned. It becomes significantly more difficult when we are trying to shift behavioral patterns of groups. This is true across a spectrum of situations and behaviors, but in this post I will focus on teachers and students shifting from a focus on content and basic skills to a focus on higher-level thinking and metacognitive skills.

These musing on “breaking the content mold” have become much more salient as I look forward to a new semester and I exchange ideas with colleagues about how we will approach our upcoming classes. I refer to the “content mold” as a way of illustrating how we, both students and teachers, have been shaped, or molded, by many years of prior experiences and expectations. Due to this shaping, the natural default for both groups is to teach or learn in ways that we have been exposed to in the past, especially if those approaches have seemed successful in the past. For many of us, this default is a focus on content and on disciplinary skills closely linked with the content. With conscious effort we can break out of that molded pattern of behavior to encourage interdisciplinary thinking and higher-level thinking skills that transfer beyond our course. However, when things get tough (e.g. when there are time constraints, high cognitive load situations, or pressures to achieve success as portrayed by exam scores), we tend to revert back to the more familiar patterns of behaviors, which for many of us means a focus on content and basic skills, rather than the use of higher-level thinking or metacognitive strategies.

Similarly, in an earlier post on this site, Ed Nuhfer points out that, “When students learn in most courses, they engage in a three-component effort toward achieving an education: (1) gaining content knowledge, (2) developing skills (which are usually specific to a discipline), and (3) gaining deeper understanding of the kinds of thinking or reasoning required for mastery of the challenges at hand. The American higher educational system generally does best at helping students achieve the first two. Many students have yet to even realize how these components differ, and few ever receive any instruction on mastering Component 3.”

One of the biggest challenges to breaking this molded pattern is that it will be far more likely to be successful if both the teacher and the student are genuinely engaged in the effort. No matter how much effort is put forth by an instructor, if value is not perceived by the student, then little change will occur. Similarly, even if a student has learned the value of higher-level thinking and metacognitive approaches, if a teacher doesn’t seem to value those efforts, then a student will astutely focus on what does seem to be valued by the teacher. A further challenge is that, over the course of a semester, the effort and motivation from both groups might wax and wane in a non-synchronous manner. As I explore these challenges, I will use myself and my less-than-successful efforts last semester as an example.

I taught an upper-level majors course in vision science, and because I have taught this course many times, I knew going in that the material is often unexpectedly challenging to students and most of them find the chapter readings to be difficult. (They contain a lot of brain biology and neural communication topics, and my students are not biology majors). Thus, I decided to build in a low-threat (with a small number of points), intentional, metacognitive reflection assignment for each lesson that had a reading. Students would indicate their level of reading completion (six levels encompassing a thorough reading with annotations, skimming, not at all) and their level of understanding of the material before class. If they had problems with any of the materials, they were supposed to indicate what steps they would take to develop understanding. They would record these and turn them in at mid-semester and at the end of the semester. I had hoped that this regular reflection would prompt their awareness of their reading behaviors and their level of learning from the reading, initiate proactive behaviors if they had poor understanding, and build habits by being completed regularly. I also took time at the start of the semester to explicitly explain why I was incorporating this regular reflection assignment.

Unfortunately, except for a couple of students, I would rate this assignment as a failure. I don’t believe it did any harm, but I also don’t believe that students used it as intended. Rather, I think most of them quickly and superficially answered the questions just so they could turn in their logs at the two required times. This type of reflection is not something that they have been asked to explicitly do in the majority (all?) of their prior courses, and they already had other strategies that seemed to work for their success in other classes For example, more than half way through the semester one student informed me that it was simply easier and faster to come to the teacher’s office and get reading guide answers (or homework problem solutions in other courses), rather than deeply read and try to figure it out on his own. Thus, if he didn’t understand as he skimmed, he didn’t worry about it. This approach wasn’t working well in my course, but up to that point he’d been very successful, so he persisted in using it (although I stopped answering his questions in my office until he could demonstrate that he’d at least tried to figure them out).

In hindsight, I believe that my actions (or lack of them) also fed into the failure. I assumed that students would bring their questions to class if they had them due to their increased awareness of them and the prompt about what they would do to increase their understanding. Thus, if there were no questions (typically the case), I used the class time to connect the readings with related application examples and demonstrations rather than reiterated what was in the readings. The students seemed engaged in class and showed no indication of specific problems with the readings. Their personal application reflection writing assignments (separate from the reading logs) were fantastic. However, their poor exam performance suggested that they weren’t deeply understanding the content, and I instinctively shifted back to my prior content-focused approaches. I also did not take time in class to directly ask them about their understanding of the readings, what parts they found most challenging, and why.

Thus, although I know I wanted to support the development of student metacognitive skills, and my students also seemed accepting of that goal when I introduced it to them at the beginning of the semester, both groups of us quickly reverted to old content-focused habits that had been “successful” in the past. I am not the first to note the challenges of developing metacognitive skills. For example, Case and Gunstone (2002) state the following, “Many … authors have emphasized that metacognitive development is not easy to foster (e.g., Gunstone & Mitchell, 1998; White, 1998). Projects to enhance metacognition need to be long-term, and require a considerable energy input from both teachers and students.”

So, what will I do in the future? My plans are to more regularly and explicitly engage in discussion of the reading reflection prompts (and other metacognitive prompts) during class. By giving class time to such discussion and bringing the metacognitive processes into the open (rather than keeping them private due to completion outside of class), I hope to indicate the value of the processes and more directly support student exploration of new ways of thinking about learning. Importantly, I hope that this more public sharing will also keep me from falling back to a simple content focus when student performance isn’t what I’d like it to be. Ultimately, metacognitive development should enhance student learning, although it is likely to take longer to play out into changed learning behaviors. I need to avoid the “quick fix” of focusing on content. Thus, I plan to shape a new mold for myself and openly display it my students. We’ll all be more likely to succeed if we are “all in” together.

——–

Nuhfer, E. (15 July 2014). Metacognition for Guiding Students to Awareness of Higher-level Thinking (Part 1). Improve with Metacognition. https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-1/

Case, J. & Gunstone, R. (2002). Metacognitive Development as a Shift in Approach to Learning: An in-depth study. Studies in Higher Education 27(4), p. 459-470. DOI: 10.1080/0307507022000011561