Heather Mitchell, Associate Professor Psychology Webster University hmitchell33@webster.edu; Kim Kleinman, Director Undergraduate Advising Webster University kleinman@webster.edu; & Ronald Daniel, Director London Webster University ronalddaniel93@webster.edu

Background / Motivation

Metacognitive practices in advising have been documented (e.g., Freeman, 2008; Sullivan-Vance, 2008), and the literature is full of suggestions on how to support purposeful student engagements and learning outcomes through advising (e.g., Campbell & Nutt, 2008; Willcoxson & Wynder, 2010). For example, academic advising is a type of student learning, and we know metacognitive approaches are one best practice to improve such learning. By infusing academic advising with such metacognitive tools, we enhance intentional student engagement through the process of advising.

Why do I have to take this class? How is this requirement going to benefit me? When will I ever use this information again? Comments similar to these three questions led us to develop the advising syllabus and curriculum planner as tools to use in academic advising. Purposeful advising is a critical component of higher education as we prepare students to be responsible, global citizens in the 21st century. Additionally, metacognition can be an extremely useful tool in an effort to promote student achievement.

Nuts and Bolts / Method

This report provides a brief overview of two advising practices (i.e., advising syllabi and curriculum planners) we use to help deliver successful, engaging experiences for students. The advising syllabus and planner are both metacognitive in nature and thus can help each student and advisor remain intentional and reflective of the student’s college career. Webster University’s advising center and individual faculty in the college refine as needed the advising syllabus for use with students. Additionally, curriculum planners, or the “Planner”, originally developed at Virginia Tech University, provide a helpful organization tool for students to use while laying out their academic path in higher education. Students at Webster University’s Geneva, Switzerland campus and students at Webster’s St. Louis campus have benefitted from these tools.

Advising Syllabus. Such a syllabus includes an advising mission or statement/ philosophy of advising and allows advisors to outline any expectations and responsibilities for both students and advisors. See Appendix A: Undergraduate Advising Syllabus as an example. Learning outcomes and a timeline / calendar of advising events are key components of such a syllabus as well as a list of resources an advisee may find useful. Advising is an essential component of an educational mission, and an advising syllabus helps specify the importance of advising similar to the way course syllabi are a regular part of every student’s classroom education. Individual advisors, either professional advisors from the University’s Advising Center or individual faculty advisors personalize the specific criteria, descriptions, learning outcomes, and responsibilities for their advisees.

The Planner. Both paper and online versions of the Planner have been created. Computer science students at Virginia Teach developed the online version of the Planner as a way of “saving” the first draft of their holistic academic plan including curricular and extra-curricular components. Both an individual student and their advisor must provide the specific knowledge and details concerning career information, interests, and plans such as graduate school, technology competences, and language competences. Students now commonly use paper versions of the Planner, which requires crucial information such as student’s course and activity interests as well as details about the availability of those courses and/or activities. See Appendix B: Planner Template.

Specifically, the Planner provides students with an opportunity to “map out” their remaining time, requirements, and other activities so students can make the most of their college experience. Students are asked to review the requirements for obtaining their specific degree and they are provided with various resources (through web links and/or Advising Worksheets appropriate to their major) to use in order to create their Planner. To complete the Planner students must include the courses taken as well as those they plan to take in their academic career. In addition to coursework, students should include any other experiences relevant to their own professional development (e.g., volunteer opportunities, research involvement, study abroad, or internships). Use of the Planner is certainly varied (similar to use of the Advising Syllabus). Advisors encourage students to plan, monitor, and evaluate their academic and co-curricular progress with these planners. Students also should include on the Planner when they might begin to search for and apply to jobs, graduate programs, etc.

Outcomes / Lessons Learned

Formal investigations of these tools have not been conducted; however, anecdotal evidence suggests students and faculty have found these tools beneficial. For example, when reflecting on The Planner, one student commented in their course evaluation [the Planner] “is a great opportunity to identify, goals and get a really actionable plan in place to achieve it.” Students appear to benefit from these tools most when students are provided ample time to understand, create, and appropriately adjust the specific mechanisms of both tools. These tools are ideal when engaged as iterative experiences. Specifically, advisors first provide appropriate scaffolding to advisees by introducing these tools. Additionally, the metacognitive nature of both tools allows students to move beyond knowing and understanding their academic requirements to analyzing, evaluating, and creating their plan to meet such requirements. In other words, the tools reflect a change from the bottom, or lower level, Bloom’s taxonomy skills to the top, or higher level, skills. The level of specific metacognitive guidance each advisor provides advisees is also completely variable as neither of these tools are mandatory. Both tools simply provide a metacognitive lens for both advisors and students to view the advising process.

References

Campbell, S.M., & Nutt C.L. (2008). Academic advising in the new global century: Supporting student engagement and learning outcomes achievement. Peer Review, 10, 4-7.

Freeman, L.C. (2008). Establishing effective advising practices to influence student learning and success. Peer Review,10, 12-14.

Sullivan-Vance, K. (2008). Reenvisioning and revitalizing academic advising at Western Oregon University. Peer Review,10, 15-17.

Wilcoxson, L., & Wynder, M. (2010). The relationship between choice of major and career, experience of university and attrition. Australian Journal of Education, 54, 175-189.

Randi Shedlosky-Shoemaker, York College of Pennsylvania, rshedlos@ycp.edu

Background

As an academic advisor for new college students, I often see them struggle to understand the demands of high school versus college, particularly in terms of self-regulation. Professors generally expect their students to complete work outside of class, including assigned and self-initiated activities (e.g., reviewing material). Given students’ experience with homework during their K-12 experience, professors and advisors may be tempted to presume college students are well practiced at completing such work and already know how to regulate related behaviors. Although research suggests that self-monitoring and knowledge of how students learn improves as they age (Brown & Smiley, 1977; Pressley, Levin, & Ghatala, 1984), college students can still struggle with this metacognitive skill (Pressley & Ghatala, 1990). The impaired metacognition may be in part due to lack of transition training.

In high school, it was likely easier for students to determine what work they had to do because someone told them what to do. College presents a different environment; perhaps for the first time, students become largely—if not solely—responsible for regulating their studying. Although students may still have assigned homework, they also have to decide what course materials to read, what to study and how, and what and when to review. This leap from being highly guided by others to being responsible for regulating their own studying can present a challenge to college students, particularly when no intermediate steps help scaffold students’ learning of self-regulation and metacognitive skills. To assist students in understanding their own study habits, I sought out to examine what role weekly study reports could play in students’ overall academic performance.

Method

I recruited undergraduate students enrolled in a mid-sized private four-year college to participate in a semester-long study on study habits. Interested students could complete up to 12 weekly study reports (adapted from Bembenutty & White; 2013; see Appendix). Through the online report, students recorded what assigned and self-initiated work they completed during the past week. Students also completed a survey at the beginning and end of the semester, measuring their feelings of motivation related to their courses (items adapted from intrinsic motivation inventory, including the choice, tension, effort, enjoyment, and value subscales; Ryan, 1982) and other academic factors (e.g., high school GPA, cumulative college GPA, credits enrolled during the current semester). In the final survey, students assessed their experience using the study reports. Students who completed both surveys and submitted at least eight weekly reports were entered into a raffle to win a gift card to the college bookstore.

Outcomes/Lessons Learned

Of the 77 students I observed during the semester, most (n = 64, 83%) submitted at least one of the 12; 36 students (47%) completed at least eight reports and 10 students (13%) submitted all 12 reports. Cumulative GPA prior to that semester was unrelated to the number of reports submitted, r(72) = 0.15, p = .21, as was academic standing based on earned credits, r(77) = -.12, p = .29. Motivation measures were unrelated to the number of reports submitted, except for choice: Students who felt they had more choice in selecting their classes also submitted more reports, r(75) = .26, p = .02.

To examine the relation to academic performance that semester, I conducted a multiple regression analysis, including previous cumulative GPA (i.e., prior to the start of the study), number of credits earned (i.e., academic standing), perceived choice in taking their courses, and number of study reports completed as potential predictors. Only two of the variables predicted semester GPA: previous cumulative GPA (B = 0.73, t = 9.54, p < .001) and number of reports submitted (B = 0.19, t = 2.33, p = .02). As previous academic success was accounted and did not predict how many study reports students completed, it seems unlikely that the positive relation between number of reports completed and semester GPA could merely be attributed to a “good student” effect, which might suggest that good students were more inclined to complete the reports, as well as engage in behaviors that improved their academic performance. Finally, none of the measures of motivation predicted whether students completed the weekly reports.

Among the 41 students who completed the final survey, most students indicated that the reports were helpful (n = 34, 83%) and felt they gained useful insight by completing the reports (n = 30, 73%). Among the open-ended remarks, students noted that the reports helped them realize how much work they were (or were not) doing and how much time tasks/classes required. Several students remarked that they had learned about their own study habits, including common distractors they struggled with, inadequate strategies they used, and their own failures in time management. In light of students’ remarks, the reports appeared to provide an opportunity to regularly and explicitly think about their own study habits. In doing so, students may develop improved metacognition related to their own learning.

If employed more intentionally as a learning tool, instructors and advisors could use the reports to go beyond helping students develop a heightened awareness of their study behaviors. Advisors could have new students or students who are struggling academically maintain a journal of their studying during a set period of time and then provide individualized suggestions to the student in an informal one-on-one conversation. Expecting students to have an actual record means that the advisor and student are not relying on autobiographical introspection of a student’s behaviors to understand current problems or develop future studying plans. In a class, particularly for courses that address effective learning strategies as a student learning outcome (e.g., first-year seminar courses, major orientation courses), instructors could create more structured engagement with the tool by not only having students record behaviors but also identify patterns of behavior over time and make evidence-based plans for future studying behaviors. Such a strategy offers an opportunity to incentivize completing the reports (e.g., course credit) and could be a valuable stepping stone for students as they transition from highly-guided learner to self-directed learner. 

References

Bembenutty, H., & White, M. C. (2013). Academic performance and satisfaction with homework completion among college students. Learning and Individual Differences, 24, 83-88. doi: 10.1016/j.lindif.2012.10.013

Brown, A. L., & Smiley, S. S. (1977). Rating the importance of structure units of prose passages: A problem of metacognitive development. Child Development, 48, 1-8. doi: 10.2307/1128873

Pressley, M., & Ghatala, E. S. (1990). Self-regulated learning: Monitoring learning from text. Educational Psychologist, 25, 19-33. doi: 10.1207/s15326985ep2501_3

Pressley, M., Levin, J. R., & Ghatala, E. S. (1984). Memory strategy monitoring in adults and children. Journal of Verbal Learning and Verbal Behavior, 23, 270-288. doi: 10.1016/S0022-5371(84)90181-6

Ryan, R. M. (1982). Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology, 43, 450-461. doi: 10.1037/0022-3514.43.3.450

Kyle E. Conlon (conlonke@sfasu.edu) and Lauren E. Brewer (brewerle@sfasu.edu), Stephen F. Austin State University

Background/Motivation: We teach psychology at a large southern university of approximately 13,000 students. Many of our students, especially freshman and first-generation students, possess ineffective study strategies that, understandably, lead to considerable frustration. They attend class, take careful notes, ask questions—do all the things we encourage them to do—and yet still underperform on their exams, leading them to ask, “What am I doing wrong?” When we ask students about their study strategies, we tend to find that they (1) rely on poor strategies (e.g., highlighting) and (2) lack insight into why their strategies aren’t working. Hence, we were motivated to create short pre- and post-exam reflection exercises to help students gain metacognitive awareness into their own study strategies.

Nuts & Bolts/Method: The pre-exam reflection exercise was designed for students to reflect on their exam preparation strategies and to identify obstacles to their studying (Appendix Table 1). The post-exam reflection exercise was designed for students to reflect on their exam performance and to determine whether it was necessary to change their study strategies for the next exam (Appendix Table 2). Fifty students (38 women, M_age = 21.10) across three psychology classes consented to participate. Each student completed four exams yielding 200 discreet observations in which a student could have completed no reflections (n = 154), pre-exam reflections only (n = 18), post-exam reflections only (n = 8), or both pre-and post-exam reflections (n = 20). For this study, we compared exam grades for students who completed both pre- and post-exam reflections to exam grades for students who completed neither pre- nor post-exam reflections. Participation was voluntary and students were informed that they could withdraw from the study at any time without penalty. In exchange for their participation, participants were entered into a raffle for one of three $100 gift cards. These gift cards were distributed at the end of the semester after final grades were submitted.

Outcomes/Lessons Learned: The exam scores of students who completed both pre- and post-exam reflections (M_grade = 86.60, SD = 11.01) were significantly higher than exam scores of students who did not complete the reflections (M_grade = 76.97, SD = 12.31) t(172) = 3.57, p < .001. Additionally, for each student the number of exam reflections completed was positively correlated with exam average (r = .42, p = .03) and with final course grade (r = .50, p = .01).

Our goal was to create brief reflection exercises to help our students gain insight into the effectiveness of their study strategies. More recently, we’ve begun to share these exercises with our academic advisees, some of whom consider dropping or avoiding classes due to poor performance. Although our specific guidance depends on the advisee, we generally encourage them to apply the exercises to the exams in the course or courses in which they’re struggling. We also try to review their responses with them to foster their metacognitive awareness (e.g., “I see you’re highlighting your notes and rereading the text; why do you think these strategies aren’t working?,” “So you felt prepared for this exam but underperformed; why do you think this happened?”). These exercises, which could be used by any academic advisor, jumpstart a discussion with advisees about how to study, which often gives them a renewed sense of hope and perspective for overcoming obstacles in their courses. In some cases, we’ll share specific articles from the metacognition literature (e.g., Putnam, Sungkhasettee, & Roediger, 2016) that dovetail with the use of these exercises. We typically meet with advisees once a semester for course selection, but we both have an open-door mentoring policy and encourage (and sometimes require) follow-up meetings with advisees, particularly those who are struggling and would benefit most from these exercises. Our experience suggests that advisees (1) generally possess poor insight into their studying (2) express surprise that their strategies aren’t as effective as they believe (or as research shows) and (3) through these exercises are forced to think through their study habits in a way they might not otherwise. We’re hopeful that improving advisees’ metacognition extends beyond the classroom to help improve their grades, motivate them beyond initial struggles, and prevent dropout.

References:

Gurung, R. A. R. (2005). How do students really study (and does it matter)? Teaching of Psychology, 32, 239–241.

Henderson, V., & Dweck, C. S. (1990). Motivation and achievement. In S. S. Feldman & G. R. Elliott (Eds.), At the threshold: The developing adolescent (pp. 308–329). Cambridge, MA: Harvard University Press.

Putnam, A. L., Sungkhasettee, V. W., & Roediger, H. L. (2016). Optimizing leaning in college: Tips from cognitive psychology. Perspectives on Psychological Science, 11, 652-660.

Trockel, M. T., Barnes, M. D., & Egget, D. L. (2000). Health-related variables and academic performance among first-year college students: Implications for sleep and other behaviors. Journal of American College Health, 49, 125–131. doi: 10.1080/07448480009596294

Mabel Ho, Department of Sociology, University of British Columbia
Katherine Lyon, Department of Sociology and Vantage One, UBC
Jennifer Lightfoot, Academic English Program, Vantage One, UBC
Amber Shaw, Academic English Program, Vantage One, UBC

Background and Motivation for Using Reflective Worksheets in Introductory Sociology

Research shows that for first year students in particular, lectures interspersed with active learning opportunities are more effective than either pedagogical approach on their own (Harrington & Zakrajsek, 2017). In-class reflection opportunities are a form of active learning shown to enhance cognitive engagement (Mayer, 2009), critical thinking skills (Colley et al., 2012), and immediate and long-term recall of concepts (Davis & Hult, 1997) while reducing information overload which can limit learning (Kaczmarzyk et al., 2013). Further, reflection conducted in class has been shown to be more effective than outside of class (Embo et al., 2014). Providing students with in-class activities which explicitly teach metacognitive strategies has been shown to increase motivation, autonomy, responsibility and ownership of learning (Machaal, 2015) and improve academic performance (Aghaie & Zhang, 2012; Tanner, 2012).

We created and implemented reflective worksheets (See Appendix) in multiple sections of a first-year sociology course at a large research university with a high proportion of international English as an Additional Language (EAL) students. While all first-year students must learn to navigate both the academic and disciplinary-specific language expectations of university, for many international students additional barriers may exist. For these students, new expectations must be achieved through their additional language and with possible diverse cultural assumptions, such as being unfamiliar with active learning and thought processes privileged in a Western academic institution. With both domestic and international students in mind, our aims with these reflective worksheets are to:

• facilitate and enhance students’ abilities to notice and monitor disciplinary awareness and knowledge while promoting disciplinary comprehension and practices.
• connect course material to personal experiences (micro) and social trends (macro).

Nuts and Bolts: Method

We structured individual writing reflection opportunities every 10-15 minutes in each lecture in the small (25 students), medium (50 students) and large (100 students) classes. Each lesson was one hour and students completed the worksheets during class time in five-minute segments. The worksheets had different question prompts designed to help students:

• identify affective and cognitive pre-conceptions about topics
• paraphrase or explain concepts
• construct examples of concepts just learned
• contrast terms
• describe benefits and limitations of social processes
• relate a concept to their own lives and/or cultural contexts
• discover connections between new material and prior knowledge (Muncy, 2014)
• summarize key lecture points (Davis & Hult, 1997)
• reflect on their own process of learning (see Appendix for further examples)

The question prompts are indicative of how to think about a topic, rather than what to think. These reflective worksheets are a way to teach students to think like disciplinary specialists in sociology, which align with the course learning outcomes. Completed worksheets were graded by Teaching Assistants (T.A.) who used the rubric below (see Table 1) to assess students’ application and critical thinking skills. By framing the worksheets as participation marks, students’ were motivated to complete the assigned work while learning how to approach sociology as a discipline. As suggested in “promoting conceptual change” (Tanner, 2012), some of the worksheets required students to recognize their preconceived notions and monitor their own learning and re-learning. For example, in one of the worksheets, students tracked their own preconceptions about a social issue (e.g. marijuana usage) in the beginning of the lecture and they returned to the same question at the end of class. Through this process, a student can have a physical record of his/her evolution of beliefs, whether it be recognizing and adjusting pre-conceived notions or deepening justifications for beliefs.

Table 1: Sample Assessment Rubric                                           

Sample Assessment Rubric
3	2	1
Entry is thoughtful, thorough and specific. Author draws on relevant course material where appropriate. Author demonstrates original thinking. Entries correspond to questions asked.	Entry is relevant but may be vague or generic. Author could improve the response by making it more specific, thoughtful or complete.	Entry is unclear, irrelevant, incomplete or demonstrates a lack of understanding of core concepts.

Outcomes: Lessons Learned

We found the reflective worksheets were effective because they gave students time to think about what they were learning and, over time, increased their awareness of disciplinary construction of knowledge. As instructors, the worksheets were a useful tool in monitoring students’ learning and ‘take away’ messages from the lectures. We also utilized the worksheets as a starting point in the next lecture to clarify any misunderstandings.

Overall, we found that while the reflective worksheets seemed to be appreciated by all the students, EAL students specifically benefitted from the worksheets in a number of ways. First, the guided questions gave students additional time to think about the topic on hand and preparation time before classroom discussion. Instead of cold-calling students, this reflective time allowed students’ to gather their thoughts and think about what they just learned in an active way. Second, students were able to explore the structure of academic discourse within the discipline of sociology. As students learn through different disciplinary lenses, these worksheets reveal how a sociologist will approach a topic. In our case, international EAL students are taking courses such as psychology, academic writing, and political science. Each of these disciplines engages with a topic using a different lens and language, and having the worksheet made the approach explicit. Last, the worksheets allow students to reflect on both the content and the way language is used within sociology. For example, the worksheets gave students time to brainstorm and think about what questions are explored from a disciplinary perspective and what counts as evidence. Furthermore, when given time to reflect on the strength of disciplinary evidence, students can then determine which language features may be most appropriate to present evidence, such as whether the use of hedges (may indicate, possibly suggest, etc.) or boosters (definitely proves) would be more appropriate. When working with international EAL students, it becomes extremely important to uncover language features so students can in turn take ownership of those language features in their own language use. Looking forward, these worksheets can help guide both EAL and non-EAL students’ awareness of how knowledge is constructed in the discipline and how language can be used to reflect and show their disciplinary understanding.

References

Aghaie, R., & Zhang, L. J. (2012). Effects of explicit instruction in cognitive and metacognitive reading strategies on Iranian EFL students’ reading performance and strategy transfer. Instructional Science, 40(6), 1063-1081.

Colley, B. M., Bilics, A. R., & Lerch, C. M. (2012). Reflection: A key component to thinking critically. The Canadian Journal for the Scholarship of Teaching and Learning, 3(1). http://dx.doi.org/10.5206/cjsotl-rcacea.2012.1.2

Davis, M., & Hult, R. E. (1997). Effects of writing summaries as a generative learning activity during note taking. Teaching of Psychology, 24(1), 47-50.

Embo, M. P. C., Driessen, E., Valcke, M., & Van Der Vleuten, C. P. (2014). Scaffolding reflective learning in clinical practice: a comparison of two types of reflective activities. Medical teacher, 36(7), 602-607.

Harrington, C., & Zakrajsek, T. (2017). Dynamic Lecturing: Research-based Strategies to Enhance Lecture Effectiveness. Stylus Publishing, LLC.

Kaczmarzyk, M., Francikowski, J., Łozowski, B., Rozpędek, M., Sawczyn, T., & Sułowicz, S. (2013). The bit value of working memory. Psychology & Neuroscience, 6(3), 345-349.

Machaal, B. (2015). Could explicit training in metacognition improve learners’ autonomy and responsibility? Arab World English Journal, 6(1), 267.

Mayer, R. E. (2009). Multimedia learning (2^nd ed.). New York, NY: Cambridge University Press.

Muncy, J. A. (2014). Blogging for reflection: The use of online journals to engage students in reflective learning. Marketing Education Review, 24(2), 101-114. doi:10.2753/MER1052-8008240202

Tanner, K. D. (2012). Promoting student metacognition. CBE-Life Sciences Education, 11(2), 113-120.

Dr. Phani Radhakrishnan & Emma Kerr
Management Department, University of Toronto
Contact: phani.radhakrishnan@utoronto.ca & emma.kerr@mail.utoronto.ca

Background/Motivation

Understanding how to seek feedback is a core topic in the curriculum in leadership courses. However, not all feedback-seeking activities are effective. The purpose of this activity is to help students to apply empirical research about feedback to their experiences in receiving and interpreting feedback. We hoped the activity would enable them to understand how to seek and interpret feedback but to also learn about how to apply it to their own learning, and thus, encourage metacognitive thinking.

Method

This activity took place approximately midway through the semester in a third-year mandatory course for students in the business administration program. There were approximately 40 students in each class. Students were introduced to the rationale for the activity by reading DeNisi and Kluger’s (2000) review article about the relation between feedback and performance. Then they answered questions requiring them to explain the key concepts in the reading and apply the theory to a real-life example (see Appendix A). Then students listened to a short lecture that explained the theory behind the factors that increase and decrease the effectiveness of feedback. The lecture focused on the finding that the way people seek feedback can have an impact on their subsequent performance. If people seek task-based feedback by looking for the correct answer or, by asking for ways to improve their answer by focusing on how to learn the task, their subsequent performance on the task will improve. But if they seek self-based feedback by looking for information on how they did they did relative to others, by seeking information for the class average their subsequent performance on the task will not improve.

Then, students reflected on how they could apply this knowledge to their own lives. They wrote a response to the following question: “Consider a situation where you got feedback and that did not help you improve your subsequent performance. Explain why the feedback was ineffective in terms of task, task learning or self-focus. What could you have done to increase your focus on task, or task learning and decrease attention to self?” Then, we gave students examples of learning goals (see Appendix B) and asked them to write learning goals with reference to the example they just wrote about. Finally, students who volunteered, read their written reflections out loud to the rest of the class. Students were graded for participation for the homework activity as well as for their participation the in-class discussion. Finally, students were asked a similar question on the final exam about feedback they received in this course. Specifically, they had to reflect on the feedback they received from an assignment in the course where they were evaluated on argumentation, definitional, data analytical, and descriptive skills and to set a learning goal on how to improve themselves on these skills. We hoped that these multiple writing prompts would encourage students to apply research about feedback to their own experiences with receiving and seeking feedback. Such written reflections should encourage students to learn about how they may help or hurt themselves by the kind of information they focus on when asking for feedback from instructors (e.g., asking for the class average vs. asking for how to improve, or asking for the correct answer).

Our activity was guided by research which suggests that providing a written prompt that encourages students to critically think about their own experiences can encourage metacognition. For example, Ratto-Parks (2015) asked first-year college students to think about a rhetorical story assignment they had completed in a course and reflect on what they did well and where they could improve. She found that student reflections improved metacognition and strengthened writing quality. Just as Ratto-Parks’ activity encouraged students to reflect on, and thus improve their writing skills, we hoped that our questions guided students on the kinds of information they should focus on while seeking feedback and also encouraged students to meta-cognize about their experiences with feedback, and to reflect on how to use feedback-seeking opportunities to improve themselves. Similarly, other studies have found that the content of written reflections that prompt critical reflection can elicit metacognitive processes (Erksine, 2009; Harten 2014; Lew & Schmidt 2010).

Further, feedback itself has also been shown to improve metacognition (Callender, 2016). In our activity, we evaluated students on multiple skills in their course-related writing assignment (e.g., argumentation, definitional skills etc.) and then asked students to reflect on what that feedback means. By asking students to relate information learned in the course to their past feedback-seeking experiences and by providing opportunities to apply that knowledge while they are getting feedback in the course, we think we are helping students to improve their metacognitive skills since they are using both written reflections and feedback as tools to develop such skills. Taken together, the in-class writing exercise, an explanation of the theory behind feedback, an opportunity to get feedback, and answering a question on the final exam about that feedback should all improve meta-cognitive skills. This is also predicted by past research cited above.

Outcomes

Preliminary analysis shows that highly engaged students (i.e., those that read the article, answered the homework questions, wrote a reflection and participated in class discussion) tended to achieve higher marks on the related final exam question. Overall, students showed an improved understanding of effective feedback following the in-class activity. We are motivated to continue to systematically analyze student responses to the initial in-class reflection questions and to the final exam questions. We hope to detect metacognitive thinking by using Ratto-Parks’ Index of Metacognitive Knowledge in Critical

Reflective Writing, which shows promise in translating metacognitive language into identifiable traits that can be used to assess students’ reflections (2015). This analysis could be challenging because our activity consists of only one in-class reflection question based on prior feedback-seeking experiences and one final exam question based on a feedback-seeking experience in the course itself. Most studies include multiple written reflections. To detect improvement in metacognition we may need to encourage students to repeatedly answer questions about what they are learning in multiple feedback contexts. This is similar to our prior research (Radhakrishnan, Arrow, & Sniezek, 1996), which shows that asking students to repeatedly evaluate their performance over multiple tests after receiving feedback on each test improves the accuracy of their evaluations. Improving students’ understanding of what they are learning, that is, their meta-cognitive skills, may also follow a similar mechanism. Multiple written reflections about how to interpret feedback while getting feedback on multiple tasks can not only help students gain an improved understanding of the theory of feedback but also about themselves.

We expect that both the improved understanding of effective feedback processes as well as the opportunity to practice metacognition will help students to interpret and give feedback more effectively both within and outside of the course. Since our students are in the management discipline, seeking feedback effectively is a skill that is essential to their professional development as leaders. In addition, we predict that the improved experience with metacognitive processes will aid them in thinking critically and interpreting feedback in their other courses as well.

References

DeNisi, A. S., & Kluger, A. N. (2000). Feedback effectiveness: Can 360-degree appraisals be improved? Academy of Management Perspectives, 14(1), 129-139. doi:10.5465/ame.2000.2909845

Erskine, D. L. (2009). Effect of prompted reflection and metacognitive skill instruction on university freshmen’s use of metacognition (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database.

Harten, M. D. (2014). An evaluation of the effectiveness of written reflection to improve high school students’ metacognitive knowledge and strategies (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database.

Lew, D. N., & Schmidt, H. G. (2011). Writing to learn: can reflection journals be used to promote self-reflection and learning? Higher Education Research & Development, 30(4), 519-532. doi:10.1080/07294360.2010.512627

Radhakrishnan, P., Arrow, H., & Sniezek, J. A. (1996). Hoping, Performing, learning, and Predicting: Changes in the Accuracy of Self-Evaluations of Performance. Human Performance, 9(1), 23-49. doi:10.1207/s15327043hup0901_2

Ratto Parks, A. E. (2015). The power of critical reflection: Exploring the impact of rhetorical stories on metacognition in first year composition stories (Doctoral dissertation).

*

Appendix A—Homework Question

The way in which feedback is given can draw one’s attention to oneself and this attention to self leads to negative effects on subsequent performance (after the feedback). However, it also discusses conditions where feedback focused on the self, may not necessarily lead to negative effects. Explain how this occurs.

Use the concept of ought vs. ideal self, and promotion vs. prevention focus. Illustrate how this theory occurs by applying it in a concrete real-life situation or example.

*

Appendix B—Examples of Learning Goals

(The following is displayed on a slide in lecture to aid students in developing their own learning goals)

For a professor…

• Finding specific ways to explain complex material in memorable ways
• Explain concepts by giving examples & counter examples
• Explain theories by giving concrete examples of process of how it works
• Show the relevance of the subject matter to the students’ lives outside the classroom

For a golfer…

• Mastering the proper grip of the club
• Master proper placement of the feet
• Learning when to use what club
• Understanding the distribution of weight from one foot to the other when swinging the club

Dr. Terrell Hooper
Assistant Professor of Music
American University of Sharjah
Email | thooper@aus.edu

Background/Motivation:

I teach an Elements of Music course for music minors and a general populous of engineering, business and architecture students needing to earn a general arts credit. I have experienced many challenges in teaching such a course in the Middle East where students have never been exposed to any elements of western music or history. The course surveys the entire gamut of western music and history, while simultaneously giving a foundational understanding of music literacy. Given the vast parameters of the course, students are expected to have strong independent study skills. While study habits are primarily individual and differ with each student, I found students not prepared or equipped with basic study skills required to be successful in the course. The most basic skill that was lacking was the ability to take notes or organize the material being discussed in class. In addition, student feedback on end of course evaluations revealed that information and material discussed in class was so unfamiliar and vast that students did not know how to organize or digest the information. From the gathered data, I inferred that students needed a note-taking model, an opportunity to take notes on their own volition, and a moment to reflect on their note taking abilities. By implementing the aforementioned objectives, I wanted to observe whether or not said objectives would encourage students to think in a metacognitive manner and would perhaps be awakened to the importance of metacognitive practices regarding their own study habits.

Method:

Due to the pedagogical “bumps” that I experienced in my first semester of teaching Elements of Music, I decided to create a sequential curriculum (see Figure 1) that would provide students with guided lecture notes[1]. The purpose of these notes were two-fold: 1) help students structure information being discussed during class 2) help students remember, reflect, and re-organize course content during independent study. The sequential curriculum was in line with the syllabus and students were not educated about note taking skills, but were merely provided with guided lecture notes that I prepared prior to each class meeting.

Three guided lecture notes were given over a three-week period (see Appendix A). Each week the guided lecture notes were designed to incorporate a progressive guide for helping students become more metacognitive aware of proper note taking habits during in-class lectures. The first guided lecture notes were designed to orient students to the process of taking notes in an outline format and contained fill in the blank areas that were curated throughout the outline. Subsequent guided lecture notes included reflective questions at the end of the lecture. Lecture 2 contained recall questions and Lecture 3 contained essay questions concerning content that was discussed during the lecture. All three guided lecture notes were collected after each class and data recorded on how many students completed the entire handout and rated on its overall completion (i.e number of blanks left on the handout). Lecture 4 (the Classical Lecture) did not use guided lecture notes and no instruction or requirements for note taking was given to students because I wanted to observe how many students saw the need to take notes of their own volition.

Following the review session (see Figure 1) a midterm exam was administered. The midterm exam consisted of multiple choice, fill in the blank, and true or false questions and were copied verbatim from the previous semester exam so data could be compared with how students not exposed to guided lecture notes scored on the same questions. After the midterm exam, students were given a survey via Google Forms and were asked questions regarding the usefulness of the guided lecture notes. Finally, I gave a ten-minute lecture that informed students on the data gathered in the questionnaire, statistics on how many students completed each handout during each lecture, and the exam scores from students who used guided lecture notes with students who did not use guided lecture notes in the Fall semester.

Figure 1. Sequential Curriculum for Guided Lecture Notes

                        Data Outcomes:

All students enrolled in Elements of Music for Spring semester participated in the study (n=29), however, due to random class absences, Lecture 1 had 27 participants, Lecture 2 had 28 participants and Lecture 3 had 25 participants. A set of 30 questions derived directly from the lecture notes were used on the midterm exam for students in Spring semester (n=29) and the final exam for students from Fall semester (n=28). Each exam question (n=30) was scored as correct or incorrect on both Fall and Spring student exams and the total number of incorrect answers was calculated for each student. An independent t-test revealed no significant difference between groups, t(53)=1.02, p=.31; Mean (Std Dev) Fall Semester = 4.9 (3.4) and Mean (Std Dev) Spring Semester = 4.0 (2.9).

On a more positive note regarding the incorporation of the guided lecture notes, students who participated in the questionnaire (n=23) gave strongly positive ratings for the notes. They rated their overall satisfaction on a 3-point Likert scale choosing between unsatisfactory, satisfactory continuum and extremely satisfactory. Results indicated 69.6% (n=15) of students surveyed were extremely satisfied with using guided lecture notes and 30.4% (n=7) of students chose the middle option, indicating neither unsatisfied nor extremely satisfied. Open-ended student feedback on using teacher guided lecture notes is represented in Table 1.

Table 1.Student Feedback Using Guided Lecture Notes

Pros	Cons
“Provides important details and helps us focus on what is more important”	“More detailed questions”
“Guides me through the chapters while studying from the book”	“Sometimes the questions are vague and need clarification”
“They were a very good guide when it came to studying for midterms as they summarized the main concepts“	“Include a list of keywords”
“These outlines make it easier to understand and absorb the material faster”	“The information was a lot and we didn’t have enough time to complete it during class while the professor was explaining it. Sometimes I felt I couldn’t keep up the pace while listening to the lecture and writing thus I left many blanks to fill in later which made me unsure of my answers.”

 Observations:

The primary purpose of this research study was to 1) help students structure information being discussed during class 2) help students remember, reflect, and re-organize course content during independent study. The study illuminated the fact that when students organize, reflect, and collaborate with their teachers on their own learning it improves the pedagogical process. Although the data does not necessarily confirm that guided lecture notes improves test scores, it would be remiss to not acknowledge that students do enjoy being provided with a structure for organizing the information presented during lectures. In addition, no negative feedback concerning the amount of material or organizational components of the course were received on end-of-course student evaluations.

The intent of helping students take personal initiative on using guided lecture notes in Lecture 4 (see Figure 1) and giving an informative ten-minute lecture on the possible gains of using such an organizational scheme when listening to class lectures was to help students to think more about their own study skills. However, generally speaking, I did not observe a change in the majority of classroom behavior with students beginning to practice metacognition regarding their own study habits. I actually observed students wanting or expecting the guided lecture notes for every class. The end-of-course student evaluations even noted that students wanted guided lecture notes for each class lecture. Even though students positively reflected on the usefulness of the guided lecture notes, I observed a disconnect in motivating students to take personal initiative for their personal study habits. Future research should investigate the link between in-class lectures and how students become more self-directed within their own independent study habits.

[1] Guided notes are defined as “teacher-prepared handouts that ‘guide’ a student through a lecture with standard cues and prepared space in which to write the key facts, concepts, and/or relationships” (Heward, 1994, p. 304).