Month: March 2017

The Strategy Project

This study presents an instructional method that requires deliberate practice of self-regulated learning strategies including active reading, management of study time and achievement goals, proactive interaction with faculty, and metacognitive reflection within the context of a student-selected course. Four instructors implemented the assignment–called “The Strategy Project”–in their first-year seminar courses, and student reflection papers were analyzed for emerging themes. These themes suggest the positive impact of applying pedagogy that requires intentional within-course application of self-regulated learning strategies, suggesting the Strategy Project may be a viable way to teach and encourage college-level strategic behavior.
For more information about this study, follow the link below:

Joining Forces: The Potential Effects of Team-Based Learning and Immediate Feedback Assessment Technique on Metacognition

by Aaron S. Richmond, Ph. D., Metropolitan State University of Denver

As a standalone assessment tool, the Immediate Feedback Assessment Technique (IF-AT) has been demonstrated to affect student learning and students’ perceptions of the teacher (e.g., Brosvic et al. 2006; Slepkov & Sheil, 2014) and possibly improve metacognition (see Richmond, 2017). However, can IF-AT be combined with a cooperative learning activity such as Team-Based Learning (TBL) to enhance metacognition as well?

To partially answer this question, several researchers suggest that the IF-AT may be used effectively with TBL (Carmichael, 2009; Hefley & Tyre, 2012; Ives, 2014). For example, you could first form teams, give them an exam to discuss and debate the correct answer, and then have the teams decide on the correct answer. If students within a team cannot come to a consensus on the response to a question, you may allow them to write an “appeal” to turn in a separate answer. Click on Figure 1 for a video on how to use IF-AT combined with TBL.  IF-AT may also be used in dyads to allow students to discuss correct and incorrect answers. Students read a question, discuss the correct and incorrect answers, and then cooperatively make a decision, with the IF-AT providing immediate feedback. A third way, suggested by Ives (2011), is to do a two-stage group quiz. Ives suggests that you should have individual students write weekly quiz questions (first-stage), then get into teams and take quizzes in teams that consist of student’s written questions. However, the question then becomes, can the combination of TBL and IF-AT instructional strategies improve metacognition?

Figure 1. Team-Based Learning Using IF-AT. 

The Interplay Among IF-AT, TBL, and Metacognition
As I argued previously (Richmond, 2017), IF-AT may improve student’s metacognition; however, by adding TBL, what metacognitive processes and skills might improve? I see a several metacognitive benefits that may occur when combining these two instructional strategies.

First, the combination of IF-AT and TBL may increase student’s metacognitive awareness. For instance, test anxiety may be reduced in a group setting when using IF-AT (Ives, 2011) because students have the opportunity to debate the answers, hear from others, gather consensus and share responsibility. As the awareness of and conscious effort to reduce test anxiety is part of metacognitive awareness, the combination of TBL and IF-AT may make this process more salient.

Second, using TBL with IF-AT may also increase student’s calibration (e.g., the accuracy of knowing when you know or do not know something). That is, in a cooperative learning activity such as TBL, students are either reinforced with their correct knowledge through the process of debating and discussion of answers OR confronted with their incorrect knowledge by interacting with team members. Consequently, their assessment (calibration) of their knowledge should become more accurate through this process. For example, if a team member accurately identifies a correct answer, and one of the team members (who had the incorrect answer to start with) observes this, they may reflect on their answer, determine why and how they came to the incorrect answer, and change future strategies to study and subsequent estimations of knowledge. Or, an individual team member could consistently get (originally) the correct answer, but always underestimate his or her knowledge. This type of student may gain confidence in their knowledge and become more accurately calibrated.

Third, by combining TBL and IF-AT, there may also be an increase of metacognitive, cognitive, and learning strategy skills. That is, as team members share how, where, what, and why they studied, other team members may incorporate these strategies  into their quiver of learning strategies (especially if the team member who suggested it was correct). For example, one team member may explain the elaborative strategy they used effectively to study, and other team members listen and incorporate elaboration into their repertoire of strategies. Or, for example, a team member may consistently get questions wrong and share what strategy he or she uses (e.g., cramming and rehearsal). Other team members observe this, realize that strategy does not appear to work very well, and subsequently rarely use it themselves (we could only wish J).

Based on the above examples, it does seem likely that the combined use of TBL and IF-AT may improve various metacognitive skills.

Concluding Thoughts and The Hallmark of Good Assessments—Evidence
As a SoTL scholar, I would be remiss not to investigate the evidence supporting or refuting the efficacy of IF-AT and TBL. There are a handful of studies that demonstrate the advantage of using TBL and IF-AT to increase academic performance and enjoyment of class (e.g., Carmichael, 2009; Haberyan, 2007). The combination of IF-AT and TBL has also demonstrated to stimulate small group discussion and identify and correct content misconceptions (Cotner, Baepler, & Kellerman, 2008). However, there appears to be a gap in the research. Specifically, there are several research questions which arise:

  1. Does the combination of IF-AT and TBL increase metacognitive awareness?
  2. Does the combination of IF-AT and TBL increase the accuracy of a student’s calibration?
  3. Does the combination of IF-AT and TBL increase a student’s repertoire of cognitive and learning strategies?
  4. What other metacognitive processes may be enhanced by using IF-AT in a TBL setting?

As I mentioned in my first blog on IF-AT (Richmond, 2017) and here, I think there are enormous SoTL research opportunities for investigating the effects of IF-AT and TBL to improve metacognition. This, invariably, leads to the proverbial phrase: A little knowledge is a dangerous thing—so get to work!

Please follow me on Twitter: @AaronSRichmond

References
Carmichael, J. (2009). Team-based learning enhances performance in introductory biology. Journal of College Science Teaching, 38(4), 54–61.

Clark, M. C., Nguyen, H. T., Bray, C., & Levine, R. E. (2008). Team-based learning in an undergraduate nursing course. Journal of Nursing Education, 47, 111–117.

Cotner, S., Baepler, P., & Kellerman, A. (2008). Scratch this! The IF-AT as a technique for stimulating group discussion and exposing misconceptions. Journal of College Science Teaching, 37(4), 48.

Haberyan, A., (2007). Team-based learning in an industrial/organizational psychology course. North American Journal of Psychology, 9,143–152.

Hefley, T., & Tyre, A. J. (2012). Favorable team scores under the team-based learning paradigm: A statistical artifact?. RURALS: Review of Undergraduate Research in Agricultural and Life Sciences, 6(1), 1. Retrieved from http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1043&context=rurals

Ives, J. (2011). Two-stage group quizzes part 1: What, how and why. Science Learnification: Teaching and learning in the sciences with a focus on physics education research (PER) from the trenches.  Retrieved from https://learnification.wordpress.com/2011/03/23/two-stage-group-quizzes-part-1-what-how-and-why/

Richmond, A. S. (2017, February 24th). Scratch and win or scratch and lose? Immediate Feedback Assessment Technique. Retrieved from https://www.improvewithmetacognition.com/scratch-win-scratch-lose-immediate-feedback-assessment-technique/

Slepkov, A. D., & Shiell, R. C. (2014). Comparison of integrated testlet and constructed-response question formats. Physical Review Special Topics-Physics Education Research, 10(2), 020120.


Does a Machine Have Metacognition?

by Roman Taraban, Ph.D.,  Texas Tech University

In the movie Arrival, the character Louise Banks is portrayed as a linguist who can decipher an alien language. For much of this task, Louise and colleagues are doing pattern matching, trying to establish a correspondence between English and the alien language. A critical piece of the plot is in the interpretation given to the translation of the alien message “offer weapon.” Louise’s competitors interpret this as “use weapon” and propose to attack and destroy the aliens. Alternatively, Louise considers whether there might be an alternative interpretation for weapon, like “tool” or “technology.” From a metacognitive perspective, we might describe the competitors as thinking at a cognitive level, interpreting the phrase literally and acting accordingly. Louise, we could say, acted metacognitively, questioning whether the literal cognitive process was sufficient. Throughout the movie, Louise questions the sufficiency of her cognitive resources for the task at hand, and figures out how to overcome her limitations. In the end, metacognition saves the day.

Normally, we think of metacognition as a value-added add-on to everyday thinking. The metacognitive individual in a sense transcends his or her own limitations. The person recognizes limitations of memory storage and speed of processing, and the value of external memory, spaced practice, planning, and so on. With this recognition of limitations of memory and processing comes a search for and discovery of strategies for managing cognition. This “higher-order” processing is metacognitive, and in the movie Arrival, Louise Banks is our metacognitive hero.

Although we are inclined to attribute metacognition to bright individuals, like Louise Banks, can we dismiss the possibility that metacognition can exist in “dumb” machines – dumb in the sense that they do not have human-like understanding? Intelligent machines, like computers, process patterns mechanically. Does a computer need to experience metacognition like a human in order for the process to be regarded as metacognitive? Is a jolt of adrenalin a necessary part of the recognition process signaling to us that we should monitor and check our calculations? The present blog is not about some distance aliens, but about a smart machine that is being widely used in many different applications today. The machine is IBM’s Watson.

There are clearly some areas in which today’s computers do not need to be metacognitive. Humans can only hold 7 + 2 chunks of information in short-term memory. An intelligent system like IBM’s Watson https://www.ibm.com/watson/developercloud/nl-classifier.html has 15 terabytes of cache memory and processes 80 teraflops per second, so neither short-term memory nor speed of processing are issues. Metacognitive processes for recognizing and preserving short-term memory would seem to be pointless, as would many of the metacognitive resource-management strategies that humans depend on. Would IBM Watson need to grab a pencil and jot a phone number onto a scrap of paper? Not likely

There may be other ways, though, that machines could exhibit metacognitive behaviors. For instance, a machine like IBM Watson might know that humans are limited in how much information they can process in a unit of time. As a metacognitive strategy, Watson might control and monitor the rate at which he verbalizes in conversation. Watson might change his linguistic register when conversing with a young child https://www.youtube.com/watch?v=vqjndtS8jQU . Watson could attempt to choose an appropriate theme with specific speakers, like Bob Dylan. In a commercial with Dylan, Watson wisely chose to talk about Dylan https://www.youtube.com/watch?v=oMBUk-57FGU. Watson apparently can monitor and modulate its own behavior depending on the context, goal, and particular interaction.

What about monitoring its own resources? If we gave Watson a set of test questions, it is not likely that Watson would reason about them metacognitively like a human. For example, Watson would not acknowledge that word problems are more difficult than straight calculations, so would attack the calculations first. However, it is not difficult to imagine situations in which Watson could reason metacognitively about his own processing and plan, control, and monitor those processes. For instance, recognizing that in the context of a crisis certain information is more critical, Watson could modify the order in which information is compiled and provided to, say, paramedics at the scene of a disaster. This would involve prioritizing specific information, queueing it up in a specific order, delivering it, and monitoring its effective transfer to the paramedics.

The irony, perhaps, is that Watson is not exhibiting “transcendent” behavior, which is how we might view metacognition in humans. Instead, Watson is simply carrying out mechanical computations, which, in a sense, are like any other computations that Watson carries out. The existence of machines like Watson should prompt us to ask whether our metacognitive ruminations may also simply be computational processes. Perhaps the sense of metacognition involving “higher-order” thinking, the self-pride we take in thinking about thinking, is an add-on, an epiphenomenon on which actual metacognitive processing in no way depends. In any case, the question of whether computers can be designed to use metacognitive strategies, to plan and modulate behaviors depending on circumstances, and to monitor the success of their efforts, may deserve a positive “yes.”