Month: June 2016

How to train reflection-in-action?

by Dr. Dominique Verpoorten, Lecturer (Learning Sciences), IFRES-University of Liège, Belgium

In this post, I’d like to introduce the notion of « reflection amplifier » in relationship with the notion of “reflection-in-action”.

Common sense tells us that reflection lies somewhere around the notion of learning and thinking. People learn as a result of reflecting. Reflection is practised in order to consider an object in more detail (Amulya, 2004).

Objects about which reflection can be amplified are innumerable. One can boost thinking on life, space, love, germs, fossils, butterflies or any content topic. This post addresses one specific topic of reflection: oneself as a learner.

Taking learning as an object of reflection is assumed to be an essential factor of expert learners (Ertmer & Newby, 1996). Reflective practice in formal learning contexts is supposed to gradually increase learners’ awareness of what helps and hampers a consistent orchestration of the various dimensions of their learning processes.

There are a number of methods that are held to encourage reflection on learning. These include learning diaries, portfolios, discussions of learning strategies, use of video and observers in a learning context, etc. These highly valuable approaches address post-practice reflection or what Schön (1983) refers to as “reflection on action”, that is a thinking episode taking place after the event and re-evaluating it so as to gain insight for improvement in the future.

But what about the training of “reflection in action”? Here come the reflection amplifiers (RAs). They present as deliberate, well-considered, and structured opportunities for students to examine and evaluate aspects of their learning experience as they occur. Unlike post-practice introspection assignments (portfolios, learning blogs…), RAs are nested in the study material and offered to individuals during learning activities. In the temporal flow of learning, their contiguity to student’s doings commits them to reflection-in-action more than to reflection on action, though Schön’s (1983) distinction is relative: even a reflection that takes place “in-action” bears on a pre-existing context. But in the case of a RA the interval is supposed to be a matter of seconds or minutes rather than hours. A typical feature of RAs is that they focus learners’ instant reflection on aspects of the learning experience they are currently committed to. RAs therefore present as brief, structured and repeated reflection affordances, interspersed in the learning material and activated during its internalization. These built-in opportunities for reflection are purposed to offer stop-and-think episodes in the course of learning. Examples of reflections amplifiers assigned to students in the course of their learning could be:

  • Rate your current mastery of the study material
  •  Give the degree of confidence you have in the correctness of your answer
  •  For one minute, evoke mentally the content at hand
  •  Give an estimation of your feeling of learning
  •  Write down a question that the teacher could ask on this topic

The word “amplifier” is used intentionally to convey the idea that enacting such affordances for reflection in the course of learning expands the mental context of the task at hand and discloses aspects of it that would otherwise be left untouched.

Reflection amplifiers have in common that they are harnessed to a first-order learning assignment. They serve it but are not confused with it due to their brevity and their meta-learning dimension. Reflection amplifiers are intended to support students at examining aspects of their learning experience in the moment of learning. They induce regular mental tinglings for evaluating “what is going on” (Salmon & al. 2007) and for nurturing internal feedback (Butler & Winne, 1995). They invite learners to think about what they are doing while they are doing it. Through establishing a practice of reflection during learning, RAs provide students with an opportunity to develop a habit of and a positive attitude towards thinking about learning.
By providing students with deliberate and structured opportunities to examine and evaluate their own learning while this learning unfolds, RAs instantiate a form of “split screen teaching” (Claxton, 2006), that consists in maintaining a dual focus on the content of the lesson and the learning dispositions and processes that are in play. Do you practice split screen teaching by sometimes providing reflection amplifiers to your students? What do they look like?

References

Amulya, J. (2004). What is Reflective Practice? Cambridge, MA: MIT – Center for Reflective Practice.

Butler, D. L., & Winne, P. H. (1995). Feedback and Self-Regulated Learning: A Theoretical Synthesis. Review of Educational Research, 65(3), 245-281.

Claxton, G. (2006, September). Expanding the capacity to learn: a new end for education? Opening keynote address presented at the British Educational Research Association Annual Conference, University of Warwick, UK.

Ertmer, P. A., & Newby, T. J. (1996). The expert learner: Strategic, self-regulated, and reflective. Instructional Science, 24(1), 1-24.
Georghiades, P. (2004). From the general to the situated: three decades of metacognition. International Journal of Science Education, 26(3), 365-383.

Salmon, P., Stanton, N., Jenkins, D., Walker, G., Young, M., & Aujla, A. (2007). What Really Is Going on? Review, Critique and Extension of Situation Awareness Theory. Engineering Psychology and Cognitive Ergonomics, 4562, 407-416.

Schön, D. (1983). The Reflective Practitioner: How professionals think in action. London: Temple Smith.

More about RAs:

http://orbi.ulg.ac.be/handle/2268/186755

http://orbi.ulg.ac.be/handle/2268/151345

http://orbi.ulg.ac.be/handle/2268/169931

http://orbi.ulg.ac.be/handle/2268/156417

http://orbi.ulg.ac.be/handle/2268/151799,

http://orbi.ulg.ac.be/handle/2268/151374


The Challenge of Deep Learning in the Age of LearnSmart Course Systems (Part 2)

A few months ago, I shared Part 1 of this post. In it I presented the claim that, “If there are ways for students to spend less time per course and still “be successful,” they will find the ways to do so. Unfortunately, their efficient choices may short-change their long-term, deep learning.” I linked this claim to some challenges that I foresaw with respect to two aspects of the online text chosen for the core course I was teaching: 1) the pre-highlighted LearnSmart text, and 2) the metacognition-focused LearnSmart quizzing feature. This feature required students to not only answer the quiz question, but also report their confidence in the correctness of that response. (See Part 1 for details to explain my concerns. Several other posts on this site also discuss confidence ratings as a metacognition tool. See references below.) My stated plan was to “regularly check in with the students, have class discussions aimed at bringing their choices about their learning behaviors into their conscious awareness, and positively reinforcing their positive self-regulation of deep-learning behaviors.” 

This post, Part 2, will share my reflections on how things turned out, along with a summary of some feedback from my students.

With respect to my actions, I did the following in order to increase student awareness of their learning choices and the impact of those choices. Twice early in the semester I took class time to explicitly discuss the possible learning shortcuts students might be tempted to take when reading the chapters (e.g. only reading the highlighted text) and when completing the LearnSmart pre-class quizzes (see Part 1 for details). I shared some alternate completion options that would likely enhance their learning and long-term retention of the material (e.g. reading the full text without highlights and using the online annotation features). Additionally, I took time to share other general learning / studying strategies that have been shown through research to support better learning. These ways of learning were repeatedly reinforced throughout the semester (and linked to content material when applicable, such as when we discussed human learning and memory).

Did these efforts impact student behaviors and choices of learning strategies? Although I cannot directly answer that question, I can share some insights based on some LearnSmart data, course performance, and reflections shared by the students.

With respect to the LearnSmart application that quizzed students at the end of each chapter, one type of data I was able to retrieve was the overall percent of time that student LearnSmart quiz question responses fell into the following correctness and confidence categories (a metacognition-related evaluation):

  1. Students answered correctly and indicated confidence that they would answer correctly
  2. Students answered correctly but indicated that they were not confident of the correctness of their response
  3. Students answered incorrectly and knew they didn’t know the answer
  4. Students answered incorrectly but reported confidence in giving the correct answer

I examined how the percentage of time student responses fell in each category correlated with two course performance measures (final exam grade and overall course grades). Category 1 (correct and confident) and Category 3 (incorrect and knew it) both showed essentially a zero relationship with performance. There was a small positive relationship between being correct but not certain (Category 2). Category 2 responses might prompt more attention to the topic and ultimately lead to better learning. The strongest correlations (negative direction) occurred for Category 4, which was the category about which I was most concerned with respect to student learning and metacognition. There are two reasons students might have responses in that category. They could be prioritizing time efficiency over learning because they were intentionally always indicating they were confident (so that if they got lucky and answered correctly, the question would count toward the required number that they had to answer both correctly and with confidence; if they indicated low confidence, then the question would not count toward the required number they had to complete for the chapter). Alternately, Category 4 responses could be due to students being erroneous with respect their own state of understanding, suggesting poor metacognitive awareness and a likelihood to perform poorly on exams despite “studying hard.” Although there was no way for me to determine which of these two causes were underlying the student responses in this category, the negative relationship clearly indicated that those who had more such responses performed worse on the comprehensive final exam and in the course at large.

I also asked my students to share some verbal and written reflections regarding their choices of learning behaviors. These reflections didn’t explicitly address their reasons for indicating high or low confidence for the pre-class quizzes. However, they did address their choices with respect to reading only the highlighted versus the full chapter text. Despite the conversations at the beginning of the semester stressing that exam material included the full text and that their learning would be more complete if they read the full text, almost half the class reported only reading the highlighted text (or shifting from full to highlighted). These students indicated that their choice was primarily due to perceived time constraints and the fact that the pre-class LearnSmart quizzes focused on the highlighted material so students could be successful on the pre-class assignment without reading the full text. More positively, a couple students did shift to reading the full text because they saw the negative impact of only reading the highlighted text on their exam grades. Beyond the LearnSmart behaviors, several students reported increasing use (even in other courses) of the general learning / study strategies we discussed in class (e.g. working with a partner to discuss and quiz each other on the material), and some of them even shard these strategies with friends!

So, what are my take-aways?

Although this should surprise no one who has studied operant conditioning, the biggest take-away for me is that for almost half my students the immediate reinforcement of being able to more quickly complete the pre-class LearnSmart quiz was the most powerful driver of their behavior, despite explicit in-class discussion and their own acknowledgement that it hurt their performance on the later exams. When asked what they might do differently if they could redo the semester, several of these students indicated that they would tell themselves to read the full text. But, I have to wonder if this level of awareness would actually drive their self-regulatory behaviors due the unavoidable perceptions of time constraints and the immediate reinforcement of “good” performance on the pre-class LearnSmart quizzes. Unfortunately, at this point, instructors do not have control over the questions asked in the LearnSmart quizzes, so that particular (unwanted) reinforcement factor is unavoidable if you use those quizzes. A second take-away is that explicit discussion of high-efficacy learning strategies can lead to their adoption. These strategies were relatively independent from the LearnSmart quiz requirement for the course, so there was no conflict with those behaviors. Although the reinforcement was less immediate, students reported positive results from using the strategies, which motivated them to keep using them and to share them with friends. Personally, I believe that the multiple times that we discussed these general learning strategies also helped because they increased student awareness of them and their efficacy (awareness being an important first step in metacognition).

————

Some prior blog posts related to Confidence Ratings and Metacognition

Effects of Strategy Training and Incentives on Students’ Performance, Confidence, and Calibration, by Aaron Richmond October 2014

Quantifying Metacognition — Some Numeracy behind Self-Assessment Measures, by Ed Nuhfer, January 2016

The Importance of Teaching Effective Self-Assessment, by Stephen Chew, Feb 2016

Unskilled and Unaware: A Metacognitive Bias, by John R. Schumacher, Eevin Akers, and Roman Taraban, April 2016


Using Metacognition to select and apply appropriate teaching strategies

by John Draeger (SUNY Buffalo State) & Lauren Scharff (U. S. Air Force Academy)

Metacognition was a recurring theme at the recent Speaking SoTL (Scholarship of Teaching and Learning) conference at Highpoint university. Invited speaker Saundra McGuire, for one, argued that metacognition is the key to teaching students how to learn. Stacy Lipowski, for another, argued for the importance of metacognitive self-monitoring through the regular testing of students. We argued for the importance of metacognitive instruction (i.e. the use of reflective awareness and self-regulation to make intentional and timely adjustments to teaching a specific  individual or group of students) as a tool for selecting and implementing teaching strategies. This post will share a synopsis of our presentation from the conference.

We started with the assumption that many instructors would like to make use of evidence-based strategies to improve student learning, but they are often faced with the challenge of how to decide among the many available options. We suggested that metacognitive instruction provides a solution. Building blocks for metacognitive instruction include 1) consideration of student characteristics, context, and learning goals, 2) consideration of instructional strategies and how those align with the student characteristics, context, and learning goals, and 3) ongoing feedback, adjustment and refinement as the course progresses (Scharff & Draeger, 2015).

Suppose, for example, that you’re teaching a lower-level core course in your discipline with approximately 35 students where the course goals include the 1) acquisition of broad content and 2) application of this content to new contexts (e.g., current events, personal situations, other course content areas). Students enrolled in the course typically have a variety of backgrounds and ability levels. Moreover, they don’t always see the relevance of the course and they are not always motivated to complete assignments. As many of us know, these core courses are both a staple of undergraduate education and a challenge to teach.

Scholarly teachers (Richlin, 2001) consult the literature to find tools for addressing the challenges just described. Because of the recent growth of SoTL work, they will find many instructional choices to choose from. Let’s consider four choices. First, Just-in-Time teaching strategies ask students to engage course material prior to class and relay those responses to their instructor (e.g., select problem sets or focused writing). Instructors then use student responses to tailor the lesson for the day (Novak, Patterson, & Gavrin, 1999; Simkins & Maier, 2004; Scharff, Rolf, Novotny, & Lee, 2013). In courses where Just-in-Time teaching strategies are used, students are more likely to read before class and take ownership over their own learning. Second, Team-Based Learning (TBL) strategies also engage students in some pre-class preparation, and then during class, students engage in active learning through a specific sequence of individual work, group work, and immediate feedback to close the learning loop (Michaelsen & Sweet, 2011). TBL has been shown to shift course goals from knowing to applying and create a more balanced responsibility for learning between faculty and students (with students taking on more responsibility). Third, concept maps provide visual representations of important connections (often hierarchical connections) between important concepts. They can help students visualize connections between important course concepts (Davies, 2010), but they require some prior understanding of the concepts being mapped. Fourth, mind mapping also leads to visual representations of related concepts, but the process is more free-form and creative, and often requires less prior knowledge. It encourages exploration of relationships and is more similar to brainstorming.

Any of these three tools might be good instructional choices for the course described above. But how is an instructor supposed to choose?

Drawing inspiration from Tanner (2012) who shared questions to prompt metacognitive learning strategies for students, we recommend that instructors ask themselves a series of questions aligned with each of our proposed building blocks to prompt their own metacognitive awareness and self-regulation (Scharff & Draeger, 2015). For example, instructors should consider the type of learning (both content and skills) they hope their students will achieve for a given course, as well as their own level of level of preparedness and time / resources available for incorporating that particular type of teaching strategy.

In the course described above, any of the four instructional strategies might help with the broad acquisition of content, and depending upon how they are implemented, some of them might promote student application of the material to new contexts. For example, while concept maps can facilitate meaningful learning their often hierarchical structure may not allow for the flexibility associated with making connections to personal context and current events. In contrast, the flexibility of mind-mapping might serve well to promote generation of examples for application, but it would be less ideal to support content acquisition. Team-Based-Learning can promote active learning and facilitate the application of knowledge to personal contexts and current events, but it requires the instructor to have high familiarity with the course and the ability to be very flexible during class as students are given greater responsibility (which may be problematic with lower-level students who are not motivated to be in the course).   Just-in-Time-Teaching can promote both content acquisition and application if both are addressed in the pre-class questions. During class, the instructor should show some flexibility by tailoring the lesson to best reach students based on their responses to the pre-class questions, but overall, the lesson is much more traditional in its organization and expectations for student engagement than with TBL. Under these circumstances, it might be that Just-in-Time strategies offer the best prospect for teaching broad content to students with varying backgrounds and ability levels.

While the mindful choice of instructional strategies is important, we believe that instructors should also remain mindful in-the-moment as they implement strategies. Questions they might ask themselves include:

  • What are you doing to “check in” with your learners to ensure progress towards daily and weekly course objectives?
  • What are signs of success (or not) of the use of the strategy?
  • How can you  adjust the technique to better meet your student needs?
  • Are your students motivated and confident, or are they bored or overwhelmed and frustrated? Are your students being given enough time to practice new skills?
  • If learning is not where it needs to be or student affect is not supportive of learning, what are alternate strategies?
  • Are you prepared to shift to them? If not, then why not?

These prompts can help instructors adjust and refine their implementation of the chosen instructional strategy in a timely manner.

If, for example, Just-in-Time assignments reveal that students are understanding core concepts but having difficulty applying them, then the instructor could tweak Just-in-time assignments by more explicitly requiring application examples. These could then be discussed in class. Alternatively, the instructor might keep the Just-in-Time questions focused on content, but start to use mind mapping during class in order to promote a variety of examples of application.  In either case, it is essential that instructors are explicitly and intentionally considering whether the instructor choice is working as part of an ongoing cycle of awareness and self-regulation. Moreover, we believe that as instructors cultivate their ability to engage in metacognitive instruction, they will be better prepared to make in-the-moment adjustments during their lessons because they will be more “tuned-in” to the needs of individual learners and they will be more aware of available teaching strategies.

While not a magic bullet, we believe that metacognitive instruction can help instructors decide which instructional strategy best fits a particular pedagogical situation and it can help instructors adjust and refine those techniques as the need arises.

References

Davies, M. (2011). Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? Higher education, 62(3), 279-301.

Michaelsen, L. K., & Sweet, M. (2011). Team‐based learning. New directions for teaching and learning,(128), 41-51.

Novak, G., Patterson, E., Gavrin, A., & Christian, W. (1999). Just-in-time teaching:

Blending active learning with web technology. Upper Saddle River, NJ: Prentice Hall.

Richlin, L. (2001). Scholarly teaching and the scholarship of teaching. New directions for teaching and learning, 2001(86), 57-68.

Scharff, L. and Draeger, J. (2015). “Thinking about metacognitive instruction” National Teaching and Learning Forum 24 (5), 4-6.

Scharff, L., Rolf, J. Novotny, S. and Lee, R. (2011). “Factors impacting completion of pre-class assignments (JiTT) in Physics, Math, and Behavioral Sciences.” In C. Rust (ed.), Improving Student Learning: Improving Student Learning Global Theories and Local Practices: Institutional, Disciplinary and Cultural Variations. Oxford Brookes University, UK.

Simkins, S. & Maier, M. (2009). Just-in-time teaching: Across the disciplines, across the

academy. Stylus Publishing, LLC.

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11(2), 113-120.