Month: May 2016

Meaningful Reflections for Improving Student Learning

by Ashley Welsh, Postdoctoral Teaching & Learning Fellow, Vantage College

I am a course coordinator and instructor for a science communication course (SCIE 113) for first-year science students. SCIE 113 focuses on writing, argumentation and communication in science and is part of the curriculum for an enriched, first-year experience program for international, English Language Learners. Throughout the term, students provide feedback on their peers’ writing in both face-to-face and online environments. The process of providing and receiving feedback is an important skill for students, however many students do not receive explicit instruction on how to provide or use constructive feedback (Mulder, Pearce, & Baik, 2014). In order to better understand my students’ experience with peer review, I conducted a research project to explore how their use and perceptions of peer review in their writing developed over the course of the term.

Many of the data collection methods I used to assess students’ perceptions and use of peer review in SCIE 113 this past term incorporated acts of reflection. These included in-class peer review worksheets and written reflections, small and large group discussions, an end-of-term survey about peer review, and my own researcher reflections. Periodically throughout the semester, I paired up the students and they engaged in peer review of one another’s writing. They each had a worksheet that asked them to comment on what their partner did well and how that person could improve their writing. During this activity, my teaching assistant and I interacted with the pairs and answered any potential questions. Afterwards, students independently completed written reflections about the usefulness of the peer review activity and their concerns about giving and receiving feedback. Before the class finished, we discussed students’ responses and concerns as a whole group. Students’ worksheets and written reflections, as well as classroom observations, offered insight into how my pedagogy mapped to their use of and reflections about peer review.

As of late, I have been more deliberate with designing pedagogy and activities that offer students the time and space to reflect and record their strengths and weaknesses as learners. The term reflection, is often used when discussing metacognition. As Weinert (1987) describes, metacognition involves second-order cognitions such as thoughts about thoughts or the reflections of one’s actions. With respect to metacognitive regulation, Zohar and Barzilai (2013) highlight that an individual can heighten their awareness of their strengths/weaknesses and evaluate their progress via reflection. This reflection process also plays a key role in metacognition-focused data collection as most methods require students to reflect upon how their knowledge and skills influence their learning. Providing survey responses, answering interview questions, and writing in a journal require a student to appraise their personal development and experience as a learner through reflection (Kalman, 2007; Aktruk & Sahin, 2011).

While the act of reflection is an important component of metacognition and metacognitive research, its use in the classroom also presents its own set of challenges. As educators and researchers, we must be wary of not overusing the term so that it remains meaningful to students. We must also be cautious with how often we ask students to reflect. An extensive case study by Baird and Mitchell (1987) revealed that students become fatigued if they are asked to reflect upon their learning experiences too often. Furthermore, we hope these acts of reflection will help students to meaningfully evaluate their learning, but there is no guarantee that students will move beyond simplistic or surface responses. To address these challenges in my own classroom, I attempted to design activities and assessments that favoured “not only student participation and autonomy, but also their taking responsibility for their own learning” (Planas Lladó et al., p. 593).

While I am still in the midst of analyzing my data, I noticed over the course of the semester that students became increasingly willing to complete the reflections about peer review and their writing. At the beginning of the term, students wrote rather simplistic and short responses, but by the end of the term, students’ responses contained more depth and clarity. I was surprised that students were not fatigued by or reluctant to complete the weekly reflections and discussions about peer review and that this process became part of the norm of the classroom. Students also became faster with completing their written responses, which was promising given that they were all English Language Learners. As per John Draeger (personal communication, April 27, 2016), students’ practice with these activities appears to have helped them build the stamina and muscles required for successful and meaningful outcomes. It was rewarding to observe that within class discussions and their reflections, students became better aware of their strengths and weaknesses as reviewers and writers (self-monitoring) and often talked or wrote about how they could improve their skills (self-regulation).

Based on my preliminary analysis, it seems that tying the reflection questions explicitly to the peer review process allowed for increasingly meaningful and metacognitive student responses. The inclusion of this research project within my class served as an impetus for me to carefully consider and question how my pedagogy was linked to students’ perceptions and ability to reflect upon their learning experience. I am also curious as to how I can assist students with realizing that this process of reflection can improve their skills not only in my course, but also in their education (and dare I say life). This research project has served as an impetus for me to continue to explore how I can better support students to become more metacognitive about their learning in higher education.

References

Akturk, A. O., & Sahin, I. (2011). Literature review on metacognition and its measurement. Procedia Social and Behavioral Sciences, 15, 3731-3736.

Baird, J. R., & Mitchell, I. J. (1987). Improving the quality of teaching and learning. Melbourne, Victoria: Monash University Press.

Kalman, C. S. (2007). Successful science and engineering teaching in colleges and universities. Bolton, Massachusetts: Anker Publishing Company, Inc.

Mulder, R.A., Pearce, J.M., & Baik, C. (2014). Peer review in higher education: Student perceptions before and after participation. Active Learning in Higher Education, 15(2), 157-171.

Planas Lladó, A., Feliu Soley, L., Fraguell Sansbelló, R.M., Arbat Pujolras, G., Pujol Planella, J., Roura-Pascual, N., Suñol Martínez, J.J., & Montoro Moreno, L. (2014). Student perceptions of peer assessment: An interdisciplinary study. Assessment & Evaluation in Higher Education, 39(5), 592-610.

Weinert, F. E. (1987). Introduction and overview: Metacognition and motivation as determinants of effective learning and understanding. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 1-16). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.

Zohar, A., & Barzilai, S. (2013). A review of research on metacognition in science education: Current and future directions. Studies in Science Education, 49(2), 121-169.


Distributed Metacognition: Insights from Machine Learning and Human Distraction

by Philip Beaman, Ph.D., University of Reading, UK

Following the success of Google’s AlphaGo programme in competition with a human expert over five games, a result previously considered beyond the capabilities of mere machines (https://deepmind.com/alpha-go), there has been much interest in machine learning. Broadly speaking, machine learning comes in two forms: supervised learning (where the machine is trained by means of examples and errors it makes are corrected) or unsupervised learning (where there is no error signal to indicate previous failures). AlphaGo, as it happens, used supervised learning based upon examples of human expert-level games and it is this type of learning which looks very much like meta-cognition, even though the meta-cognitive monitoring and correction of the machine’s performance is external to the system itself, although not necessarily to the machine which is running the system. For example: an artificial neural network (perhaps of the kind which underpins AlphaGo) is trained to output Y when presented with X by means of a programme which stores training examples – and calculates the error signal from the neural network’s first attempts – outside the neural network software itself but on the same hardware. This is of interest because it illustrates the fluid boundary between a cognitive system (the neural network implemented on computer hardware) and its environment (other programmes running on the same hardware to support the neural network) and demonstrates that metacognition, like first-order cognition, is often a form of situated activity. Here, the monitoring and the basis for correction of performance is (like all supervised learning) external to the learning system itself.

In contrast, when psychologists talk about metacognition, we tend to assume that all the processing is going on internally (in the head), whereas in fact it is usually only partly in the head and partly in the world. This is not news to educationalists or to technologists: learners are encouraged to make effective use of external aids which help manage work and thought, but external aids to cognition are often overlooked by psychological theories and investigations. This was not always the case. In the book “Plans and the Structure of Behaviour” which introduced the term “working memory” to psychology, Miller, Galantner and Pribram (1960) spoke of working memory as being a “special state or place” used to track the execution of plans where the place could be in the frontal lobes of the brain (a prescient suggestion for the time!) or “on a sheet of paper”. This concept that was originally defined wholly functionally has, in subsequent years, morphed into a cognitive structure with a specific locus, or loci, of neural activity (e.g., Baddeley, 2007; D’Esposito, 2007; Henson, 2001; Smith, 2000).

We have come across the issue of distributed metacognition in our own work on auditory distraction. For many years, our lab (along with several others) collected and reported data on the disruptive effects of noise on human cognition and performance. We carefully delineated the types of noise which cause distraction and the tasks which were most sensitive to distraction but – at least until recently – neither we nor (so far as we know) anyone else gave any thought to meta-cognitive strategies which might be employed to reduce distraction outside the laboratory setting. Our experiments all involved standardized presentation schedules of material for later recall and imposed environmental noise (usually over headphones) which participants were told to ignore but which they could not avoid. The results of recent studies which both asked participants for their judgments of learning (JoLs) concerning the material and gave them the opportunity to control their own learning or recall strategy (e.g., Beaman, Hanczakowski & Jones, 2014) are of considerable interest. Theoretically, one of three things might happen: meta-cognition might not influence ability to resist distraction in any way, meta-cognitive control strategies might ameliorate the effects of distraction, or meta-cognition might itself be affected by distraction potentially escalating the disruptive effects. For now, let’s focus on the meta-cognitive monitoring judgments since these need to be reasonably accurate in order for people to have any idea that distraction is happening and that counter-measures might be necessary.

One thing we found was that people’s judgments of their own learning was fairly well-calibrated, with judgements of recall in the quiet and noise conditions mirroring the actual memory data. This is not a surprise because earlier studies, including one by Ellermeier and Zimmer (1997) also showed that , when asked to judge their confidence in their memory, people are aware of when noise is likely to detract from their learning. What is of interest, though, is where this insight comes from. No feedback was given after the memory test (i.e., in neural network terms this was not supervised learning) so it isn’t that participants were able to compare their memory performance in the various conditions to the correct answers. Ellermeier and Zimmer (1997) included in their study a measure of participants’ confidence in their abilities before they ever took the test and this measure was less well calibrated with actual performance so this successful metacognitive monitoring does seem to be dependent upon recent experience with these particular distractors and the particular memory test used, rather than being drawn from general knowledge or past experience. What then is the source of the information used to monitor memory accuracy (and hence the effects of auditory distraction on memory)? In our studies, the same participants experienced learning trials in noise and in quiet in the same sessions and the lists of items they were required to try and recall were always of the same set length and recalled by means of entering into a physical device (either writing or typing responses). Meta-cognitive monitoring, in other words, could be achieved in many of our experiments by learning the approximate length of the list to be recalled and comparing the physical record of number of items recalled with this learned number on a trial-by-trial basis. This kind of meta-cognitive monitoring is very much distributed because it relies upon the physical record of the number of items recalled on each trial to make the appropriate comparison. Is there any evidence that something like this is actually happening? An (as yet unpublished) experiment of ours provides a tantalising hint: If you ask people to write down the words they recall but give one group a standard pen to do so and another group a pen which is filled with invisible ink (so both groups are writing their recall, but only one is able to see the results) then it appears that monitoring is impaired in the latter case – suggesting (perhaps) that meta-cognition under distraction benefits from distributing some of the relevant knowledge away from the head and into the world.

References:

Baddeley, A. D. (2007). Working memory, thought and action. Oxford: Oxford University Press.

Beaman, C. P., Hanczakowski, M., & Jones, D. M. (2014). The effects of distraction on metacognition and metacognition on distraction: Evidence from recognition memory. Frontiers in Psychology, 5, 439.

D’Esposito, M. (2007) From cognitive to neural models of working memory. Philosophical Transactions of the Royal Society B: Biological Sciences, 362, 761-772.

Ellermeier, W. & Zimmer, K. (1997). Individual differences in susceptibility to the “irrelevant sound effect” Journal of the Acoustical Society of America, 102, 2191-2199.

Henson, R. N. A. (2001). Neural working memory. In: J. Andrade (Ed.) Working memory in perspective. Hove: Psychology Press.

Miller, G. A., Galanter, E. & Pribram, K. H. (1960). Plans and the structure of behavior. New York: Holt.

Smith, E. E. (2000). Neural bases of human working memory. Current Directions in Psychological Science, 9, 45-49.


Learning to Write and Writing to Learn: The Intersection of Rhetoric and Metacognition

by Amy Ratto Parks, Ph.D., University of Montana

If I had to choose the frustration most commonly expressed by students about writing it is this: the rules are always changing. They say, “every teacher wants something different” and because of that belief, many of them approach writing with feelings ranging from nervous anxiety to sheer dread. It is true that any single teacher will have his or her own specific expectations and biases, but most often, what students perceive as a “rule change” has to do with different disciplinary expectations. I argue that metacognition can help students anticipate and negotiate these shifting disciplinary expectations in writing courses.

Let’s look at an example. As we approach the end of spring semester, one single student on your campus might hold in her hand three assignments for final writing projects in three different classes: literature, psychology, and geology. All three assignments might require research, synthesis of ideas, and analysis – and all might be 6-8 pages in length. If you put yourself in the student’s place for a moment, it is easy to see how she might think, “Great! I can write the same kind of paper on three different topics.” That doesn’t sound terribly unreasonable. However, each of the teachers in these classes will actually be expecting some very different things in their papers: acceptable sources of research, citation style and formatting, use of the first person or passive voice (“I conducted research” versus “research was conducted”), and the kinds of analysis are very different in these three fields. Indeed, if we compared three papers from these disciplines we would see and hear writing that appeared to have almost nothing in common.

So what is a student to do? Or, how can we help students anticipate and navigate these differences? The fields of writing studies and metacognition have some answers for us. Although the two disciplines are not commonly brought together, a close examination of the overlap in their most basic concepts can offer teachers (and students) some very useful ways to understand the disciplinary differences between writing assignments.

Rhetorical constructs are at the intersection of the fields of writing studies and metacognition because they offer us the most clear illustration of the overlap between the way metacognitive theorists and writing researchers conceptualize potential learning situations. Both fields begin with the basic understanding that learners need to be able to respond to novel learning situations and both fields have created terminology to abstractly describe the characteristics of those situations. Metacognitive theorists describe those learning situations as “problem-solving” situations; they say that in order for a student to negotiate the situation well, she needs to understand the relationship between herself, the task, and the strategies available for the task. The three kinds of problem-solving knowledge – self, task, and strategy knowledge – form an interdependent, triangular relationship (Flavell, 1979). All three elements are present in any problem-solving situation and a change to one of the three requires an adjustment of the other two (i.e., if the task is an assignment given to whole class, then the task will remain the same; however, since each student is different, each student will need to figure out which strategies will help him or her best accomplish the task).

Metacognitive Triangle

The field of writing studies describes these novel learning situations as “rhetorical situations.” Similarly, the basic framework for the rhetorical situation is comprised of three elements – the writer, the subject, and the audience – that form an interdependent triangular relationship (Rapp, 2010). Writers then make strategic persuasive choices based upon their understanding of the rhetorical situation.
Rhetorical vs Persuasive

 In order for a writer to negotiate his rhetorical situation, he must understand his own relationship to his subject and to his audience, but he also must understand the audience’s relationship to him and to the subject. Once a student understands these relationships, or understands his rhetorical situation, he can then conscientiously choose his persuasive strategies; in the best-case scenario, a student’s writing choices and persuasive strategies are based on an accurate assessment of the rhetorical situation. In writing classrooms, a student’s understanding of the rhetorical situation of his writing assignment is one pivotal factor that allows him to make appropriate writing choices.

Theorists in metacognition and writing studies both know that students must be able to understand the elements of their particular situation before choosing strategies for negotiating the situation. Writing studies theorists call this understanding the rhetorical situation while metacognitive theorists call it task knowledge, and this is where two fields come together: the rhetorical situation of a writing assignment is a particular kind of problem-solving task.

When the basic concepts of rhetoric and metacognition are brought together it is clear that the rhetorical triangle fits inside the metacognitive triangle and creates the meta-rhetorical triangle.

Meta-Rhetorical Triangle

The meta-rhetorical triangle offers a concrete illustration of the relationship between the basic theoretical frameworks in metacognition and rhetoric. The subject is aligned with the task because the subject of the writing aligns with the guiding task and the writer is aligned with the self because the writerly identity is one facet of a larger sense of self or self-knowledge. However, audience does not align with strategy because audience is the other element a writer must understand before choosing a strategy; therefore, it is in the center of the triangle rather than the right side. In the strategy corner, however, the meta-rhetorical triangle includes the three Aristotelian strategies for persuasion, logos, ethos, and pathos (Rapp, 2010). When the conceptual frameworks for rhetoric and metacognition are viewed as nested triangles this way, it is possible to see that the rhetorical situation offers specifics about how metacognitive knowledge supports a particular kind problem-solving in the writing classroom.

So let’s come back to our student who is looking at her three assignments for 6-8 papers that require research, synthesis of ideas, and analysis. Her confusion comes from the fact that although each requires a different subject, the three tasks are appear to be the same. However, the audience for each is different, and although she, as the writer, is the same person, her relationship to each of the three subjects will be different, and she will bring different interests, abilities, and challenges to each situation. Finally, each assignment will require different strategies for success. For each assignment, she will have to figure out whether or not personal opinion is appropriate, whether or not she needs recent research, and – maybe the most difficult for students – she will have to use three entirely different styles of formatting and citation (MLA, APA, and GSA). Should she add a cover page? Page numbers? An abstract? Is it OK to use footnotes?

These are big hurdles for students to clear when writing in various disciplines. Unfortunately, most faculty are so immersed in our own fields that we come to see these writing choices as obvious and “simple.” Understanding the way metacognitive concepts relate to rhetorical situations can help students generalize their metacognitive knowledge beyond individual, specific writing situations, and potentially reduce confusion and improve their ability to ask pointed questions that will help them choose appropriate writing strategies. As teachers, the meta-rhetorical triangle can help us offer the kinds of assignment details students really need in order to succeed in our classes. It can also help us remember the kinds of challenges students face so that we can respond to their missteps not with irritation, but with compassion and patience.

References

Flavell, J.H. (1979). Metacognition and cognitive monitoring: A new era cognitive development inquiry. American Psychologist, 34, 906-911.

Rapp, C. (2010). Aristotle’s Rhetoric. In E. Zalta (Ed.), The stanford encyclopedia of

philosophy. Retrieved from http://plato.stanford.edu/archives/spr2010/entries/aristotle-rhetoric/