Month: January 2021

Learning. Design. Analytics. Post 1: A Faculty Development Approach To Support Metacognitive Awareness During Course Adaptation

By Yianna Vovides, PhD (Series Editor), Georgetown University

I once worked with a faculty member who was skeptical about teaching an online course, let alone spending time working with a designer on it. So, after my first meeting with him, realizing his hesitations, I created a prototype based on his course syllabus to show him what was possible. I remember him saying, I couldn’t see how to teach my course online, I am not a techie, but maybe I can if you help me.

He now saw me as his coach and partner, helping him plan how to engage students, helping him put in place assignments that he could manage within the course management system, helping him during his teaching. All along, during the four months we spent on his course, I would ask him about his teaching philosophy and his approach to teaching in his discipline. About a month before the course was ready to launch, I asked him if he could write a few paragraphs to explain to his students what he was sharing with me about his choices in the readings, his expectations in relation to how students approached a text and what he looked for in their assignments. He did.

We ended up recording these (only audio) and adding them to his week-by-week course structure. I then asked him if he was up for doing some more recording that focused on the selection of texts in his courses. I asked him to share his study of the authors themselves. He did. I then created an e-book that students would use to explore a bit more about the authors from their instructor’s perspective.

When the course opened, I spent an hour on the phone walking him through how to respond to student posts in the discussion board. He said, Thank you, I think I can do this! And he did. During the first run of his course, I sent him weekly emails to check in and point out the student monitoring/analytic features for making sure his students were keeping up.

What does metacognition have to do with it?

Because the process of online course development takes time, the relationship between the designer and the faculty tends to result in one that lasts past that one course experience. It is usually after the first course design and the first time faculty teach their course when they realize how much they learned about teaching and learning. They then go on to adapt their other courses. They are more metacognitively aware. They are aware of their own approaches to teaching and learning, aware of what it takes to design and teach a course in another mode, and are aware that good design and teaching involves planning, monitoring, reflecting, evaluating, and adapting existing practices. This is how I define the process of course adaptation that we will explore further in this post.

Let us dig a bit deeper into course adaptation.

In this post, I describe the adaptive approach we have implemented as part of our online programs efforts at the Center for New Designs in Learning and Scholarship (CNDLS), Georgetown University. The approach connects instructional design practices with a faculty development focus that encourages metacognition (planning, monitoring, evaluating). I started with the following overarching question: How should instructional designers guide faculty to rethink their approach to course design to follow an adaptive faculty development process? I then identified the following sub-questions that formed the basis of the approach and operationalizing the process: 

  • What techniques can instructional designers follow to engage faculty in design thinking? 
  • What techniques can instructional designers follow to engage faculty in meta reflections about their teaching methods? 
  • How can instructional designers use learning analytics to help faculty continue engaging in meta reflections during their online teaching? 

The questions I listed above offered the CNDLS online programs team a way to problematize our approach to design. I realized that we needed to make visible the levels (macro, meso, micro) that we address during the design process and enable faculty to navigate these successfully. We implemented a model that enables conversations about design and development at all levels by following a before, during, after approach (see Figure 1).

circle schematic with three equal components: before, during, after
Figure 1. Macro – Before, during, after model

The guiding questions start at the course level and move to sessions and sequence of engagement exploring the teaching and learning experience across time. These questions include but are not limited to the following:

  • Tell us about your course. What do you love about this course? What do you think the students love about this course?
  • What are the things that you think about when you prepare to teach this course?
  • How do you engage your students before the semester starts?
  • What do you do during that first class session?
  • What do you expect students to do during the first class session?
  • What do you do after the class session?
  • What do you want students to do after the class session?

These questions help faculty reflect about their approach to teaching and learning. By asking these questions up front and throughout the course adaptation process we are embedding metacognitive instruction within the course design model itself. In addition, throughout the design process we include check-in sessions that allow both the designer and faculty to pause and ask:

  • Is our design plan still valid?
  • Is our choice of technology going to support students in their learning process?
  • Do we need to do anything differently?

What these check-in questions do over the span of four to six months of engaging with an individual faculty on the course design and development process is that the conversations become connected across time and merge into a spiral design model. Figure 2 visualizes the spiral model that supports the faculty development approach that instructional designers take. Once faculty members experience this model, they continue to follow this design approach as they envision their other courses. In addition, they tend to re-visit their approach to their teaching shifting from an instructor-centered to a student-centered approach.

Schematic of a spiral illustrating loops of before, during, after
Figure 2. Spiral – Before, during, after Model

Because the model is based on time, it easily communicates across the various disciplines. What do I mean by that? Because the conversations that surround this model are related to teaching practices, it is also a way to account for contact time (faculty-student interaction) and learning time (student effort). We refer to the combination of contact time and learning time as instructional time in conversations with faculty. In remote teaching and learning, instructional time is an entry point to envisioning how learning can happen in different ways.

The rest of the mini-series on Learning. Design. Analytics. includes examples using this approach that highlight strategies used to activate metacognitive awareness during the course design and re-design process through the designer-faculty interaction. In addition, the series highlights how technology interventions and learning analytics are integrated as part of the process.

Some background about instructional design and online education to frame the approach

Adapting traditional classroom-based courses to online may sound simple given that online education has been around for more than two decades. In fact, instructional design, a field of study that is over 80 years old, offers theories, models, and processes that guide designers to make this adaptation from traditional classroom-based teaching to online. This is a technical challenge – solutions are available and are knowable. However, in higher education, the instructional design process, when framed to support faculty development, introduces complexity. The challenge is no longer technical because the focus of the challenge is no longer about the course adaptation from traditional to online but the people involved in making the adaptation happen (faculty, designers, media specialists, students, and other members of the team that supports this process). It is a process of transformation.

Let us pull this apart a bit more. Higher education as an institution has been described as lacking innovation and flexibility for promoting impactful teaching and learning (Rooney et al., 2006). That was in 2006. Between 2006 and 2016,  we have seen online education grow and thrive with over 6 million students (approximately 30% of all higher education students) enrolled in at least one distance education course in the United States (Allen & Seaman, 2017). Then COVID-19 happened. Remote teaching and learning is happening across the globe and is now the new normal. Given the speed of the changes, some schools have been able to pivot and put in place the needed support for their instructors while others are struggling to determine what that support needs to be and how to operationalize it. 

There are many factors that contribute to these decisions besides resources such as institutional, departmental, and individual cultural norms. For example, the institutional culture may be known by those who are in it but much of it is hidden from those new to it which may lead to actions that are oftentimes driven by assumptions rather than visible evidence (Halupa, 2019). Many academic departments tend to value individual contributions and can propagate a competitive rather than a collaborative environment. This may then lead to a less cohesive curriculum online. Individual faculty members are experts in their discipline but not necessarily in the discipline of teaching and learning. Therefore, within this complex network of needs, faculty development efforts in higher education try to balance group and individual engagements to provide opportunities for faculty to get the support they need in their teaching.

Recognizing that there are different instructional development needs necessitates that we offer different entry points and pathways in our faculty development programming. Within the online course design efforts, we work with faculty to help them see their teaching challenge from a design thinking perspective that begins with an exploration of what individual learners will experience. By doing so, we are no longer facing a technical challenge but rather an adaptive one because we are now focusing on individual learner needs. To tackle this adaptive challenge that is implicitly dynamic because of the focus on humans, we argue that the approach requires that planning, monitoring, and evaluation become an integral part of the process at both the cognitive and metacognitive levels. 

References

Allen, I. E., & Seaman, J. (2017). Digital Compass Learning: Distance Education Enrollment Report 2017. Babson survey research group.

Halupa, C. (2019). Differentiation of Roles: Instructional Designers and Faculty in the Creation of Online Courses. International Journal of Higher Education, 8(1), 55-68.

Rooney, P., Hussar, W., Planty, M., Choy, S., Hampden-Thompson, G., Provasnik, S., & Fox, M. A. (2006). The Condition of Education, 2006. NCES 2006-071. National Center for Education Statistics.


Using Metacognition to Facilitate Scholarly Identity

by Anton Tolman, Ph.D., Guest Editor

This is the final and concluding blog for our series. I want to thank my colleagues for their time and effort in this project: Steven Pearlman, Christopher Lee, and Benjamin Johnson. Speaking for all of us, we hope you found our thoughts helpful in enriching your own thinking regarding metacognition and its importance in student learning.

The topics of this series, critical thinking, inclusive classrooms, student motivation, and succeeding with collaborative learning, are all essential themes in local and national discussions right now concerning student engagement and effective teaching. Each of the blogs in the series also touched on resistance to change (faculty or student), either explicitly or implicitly, the role of humility, and the development of metacognitive skills in achieving successful outcomes. Enhancing metacognition in ourselves and in our students is an ongoing progression, a journey, and we are happy to be walking it with you. In this last blog, I address the connection between metacognition and development of students’ personal narrative, their identity as scholars or educated persons. I believe this is the true heart of higher education and the core of its value to society.

photo of a blindfolded business man reaching forward and a big forward-facing arrow painted on the ground in front of him

You Can’t Change What You Can’t See

This phrase is an axiom in clinical work with clients. Clients often come to see a therapist knowing that things are wrong in their lives, but they don’t understand the reasons why or do not see a path forward to healing. They can’t change their lives for the better until they begin to “see” the nature of their problems, accept responsibility for their own role in those problems, and imagine and start to walk the road ahead.

This axiom applies to students. They usually come to college based on the promise of an economic benefit like higher paying jobs, or because they see a degree as a requirement for future goals. Many, if not most, see the purpose of education as learning facts or information and therefore, see the role of professors as experts who teach them content. When they are confronted with assignments that ask them to use critical thinking, solve problems, or work together, they can become easily frustrated. Thus, the terms “jumping through hoops” and “busy work” are commonly found in student conversations about their classes. These forms of resistance (Tolman & Kremling, 2017) are understandable because many students can’t see that the real goal of higher education is skill development, not content; it is not easily visible to them. Like the therapist’s clients, they won’t make progress until they develop the capacity to recognize the underlying issues and see the path ahead as one of purpose and value.

Student resistance to learning begins to diminish when students evaluate their own attitudes and behaviors and connect those behaviors to their academic performance. When they learn to develop metacognitive skills they can “see” previously unseen patterns in themselves and others: they recognize their own complicity in their academic struggles and begin to grasp that they are not just consumers of external information or persons being judged by some authority figure. This empowers them to assume responsibility, take action on their own, to succeed, to grow, and to become part of a community of learners.

wireframe image of a human head facing forward wit blue points like starts surrounding it.

Seeing is Believing: Shifting Identity in Higher Education    

In his recent blog, Taraban (2020) describes identity as an ongoing form of development grounded in episodic memory: the story we tell ourselves about who we are. This self-narrated story is strongly shaped by the boundaries of what students “see” as the purpose of education, their personal goals, and how they approach learning. If students’ sense of identity about who they are does not change from that of being consumers of content or “students”, then we have failed them.

If we were to adopt the model of cognitive apprenticeship (Collins, Brown, & Newman, 1987) in our teaching, seeing ourselves more as mentors to students, then our major task becomes to shift their story, their identity, to that of being apprentices, not students. As apprentices, they are learning new skills under the guidance of an expert who cares for them, and who asks them to constantly re-evaluate what they are learning, consider how they are learning it, and when and how to use what they are learning. This entails a transition towards seeing themselves as participating members of the academy, as scholars and educated persons who contribute to society; metacognition is at the core of this identity shift.

Undergraduate research is a great example of this as articulated by Charity-Hudley, Dickter, and Franz (2017). They explain that the mechanism of action of this “high impact” practice on student success and retention, especially for minority or under-represented students, begins as students enter into a mentored relationship with a professor. Moving away from the traditional “student” role enables them to realize there is more to learning besides getting a grade or completing course assignments. Metacognitive activity like learning to reflect and ask their own questions, carry out their own research, generate new data, challenge their own ideas as well as existing ideas in the discipline, and create new understandings, makes them a contributor to knowledge, not just a consumer. They are a scholar, or at least a scholar-apprentice, and those episodic memories begin to shift their own narrative identity — who they see themselves to be, how they interpret their own life and future. Of course, participating in research is not the only path available to this outcome.

This student progression requires creating opportunities for students to develop and use their metacognitive skills. In both Steve Pearlman’s (this blog series) and Hale’s (2012) potent arguments, the development of metacognitive and critical thinking skills is integral to development of a “personal intellectual narrative”; you cannot discuss metacognition without referencing aspects of critical thinking, and you cannot explain critical thinking without referring to the metacognitive processes involved. As Hale (2012) says, cultivation of an “intellectual language” is a key process in this development; it inducts students into the “Great Conversation” and it becomes part of their own personal history of intellectual development. The more we integrate metacognitive opportunities in our classes, and across the curriculum, the more likely we are to observe this transition occur.

Suggestions for Teaching

Here are some thoughts about ways to incorporate metacognitive practices that promote personal narratives in students:

  • Emphasize transparency and relevance. Explain the purposes of our assignments not just for short-term outcomes (learn something for a grade), but for the long-term (learn something to enhance a career and personal life, contribute to society); define and set expectations about the value of metacognition and its role in professional thinking within your discipline.
  • Assign metacognitive tasks that require students to evaluate their strengths and weaknesses as learners, identify learning strategies they are using and those they are not, and ask them to connect this information to their personal and career goals. Benjamin Johnson’s description of the Personal Learning Plan (this blog series), based partly on completing practical metacognitive inventories and evaluating how to improve is an example.
  • Emphasize the value of students thinking about their own development over time and their personal histories; reflective writing assignments, in all fields, are useful for this.
  • Use inquiry assignments requiring students to develop their own questions, do their own research, and apply it to course content and their lives.
  • Create opportunities for students to make their own thinking visible to themselves. Encourage them to question their learning, their assumptions, and acknowledge their areas of confusion as a community of learners. Hale (2012) suggests learning logs, real-time student writing of their thinking, questions, and descriptions of how they are approaching content, assignments, and preparation.
  • Shift your role from “sage on the stage” to a mentor of cognitive apprentices. Model professional thinking; demonstrate metacognition and critical thinking and help the students recognize it and practice it. One way I do this is to ask, and continuously reinforce, that students call me Coach T. In my syllabus I explain the rationale for this: my purpose is to facilitate their learning, give them exercises to improve, and to clarify or assist, but the basic responsibility for their learning, as with any athlete, actor, or musician, lies with themselves.
  • Evaluate your course design: what are the memories and personal experiences your students will take away relevant to metacognition? Do your assignments focus primarily on content acquisition or do they promote skill development, a sense of growth and progress towards becoming a scholar, ability to speak the intellectual language of the discipline and to reason within its context? What are your course objectives and where do they point your students: towards content, or towards becoming scholars?

photo of a woman peeking out from under a black blindfold

These teaching practices help students “connect the dots” and see patterns they did not know existed: how they approach learning, how well they are learning, the purpose of education, and their own intellectual growth and development. Doing this reduces resistance and shifts their understanding of learning and of themselves. When we move our perspective from content to skills and weave metacognitive development into the fabric of our class, we create an environment encouraging the exploration of new personal narratives and identity for our students. This brings us closer towards achieving the potential that higher education has to offer. If you are already doing these things, hone your work, expand your empathy, and become more transparent. If you are not, you can see the road ahead, and you don’t have to travel it alone. Reach out, learn from others, and find greater joy in what you do.

References:

Charity-Hudley, A.H., Dickter, C.L., & Franz, H.A. (2017). The Indispensable Guide to Undergraduate Research: Success In and Beyond College. New York: Teachers College Press.

Collins, A., Brown, J. S., & Newman, S. E. (1987). Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics (Technical Report No. 403). BBN Laboratories, Cambridge, MA.

Hale, E. (2012). Conceptualizing a personal intellectual history/narrative: The importance of strong-sense metacognition to thinking critically. In M.F. Shaughnessy (Ed). Critical Thinking and Higher Order Thinking. Nova Science Publishers, Inc.

Taraban, R. (2020, June 25). Metacognition and the Development of Self. ImproveWithMetacognition.com. https://www.improvewithmetacognition.com/metacognition-and-self-identity/

Tolman, A.O. & Kremling, J. (2017). Why Students Resist Learning: A Practical Model for Understanding and Helping Students. Sterling, VA: Stylus Publishing.


Boosting the Effectiveness of Collaborative Learning Using Metacognition

by Anton Tolman, PhD, Guest Editor, Utah Valley University

The introductory blog for this series included a figure of the Integrated Model of Student Resistance (IMSR; Tolman & Kremling, 2017). That image (shown below) illustrates that student resistance is the outcome of systemic factors, including lack of metacognition. The IMSR demonstrates that family and the larger culture (including institutional culture) may promote consumer expectations and that these elements (poor metacognitive skills and culture) often intersect with instructor behaviors and attitudes, leading to negative experiences in education. This, in turn, increases student resistance, especially towards active learning.

These elements are also exacerbated by student cognitive development, which is overlooked by many instructors as a relevant element in metacognition, classroom experiences, and resistance. For example, Nufher (2014) summarizes the work of William Perry (1999) who carried out longitudinal studies of college student cognitive development. Students react with varying degrees of resistance to education depending on their level of cognitive development. Progressing in cognitive development can take time, but several authors have noted that effective use of metacognition and other strategies may accelerate this process, including careful metacognitive exercises (see Nuhfer, 2014; Nelson, 2015; Kloss, 1994).

The IMSR was created to help instructors begin to see resistance differently than they typically do; it is a communication signal from students that something is not working and, rather than dismiss it or try to assert their authority, the solution lies in addressing the underlying causes, including lack of metacognition. A recently published study in nursing (Stover & Holland, 2018) reported that use of the IMSR to redesign a collaborative learning-based course resulted in reduced resistance to collaborative learning, a greater sense of belonging to a community of inquiry, higher student satisfaction, and less negative comments or concerns. For a full explanation of the model, see Tolman and Kremling (2017).

flow chart showing components of the Integrated_Model_of_Student_Resistance

Students not actively using metacognitive skills are more likely to resist active teaching efforts because they see themselves as passive consumers of information whose main concern is to meet requirements set by an authority figure in order to graduate. My experience is that student resistance is greater in courses requiring collaborative learning because students are expected to work together on activities important to their grade. Now, imagine the level of resistance to collaborative learning in my online Abnormal Psychology course! When students see the syllabus, the most common resistant comment I receive is “I took this course online so I would not have to work with others.”

We should acknowledge that some of that resistance is justified. Collaborative learning pedagogies are not simple, and unfortunately are sometimes implemented ineffectively, leading to negative experiences for students. U.S. culture is highly individualistic and emphasizes personal success rather than group efforts despite the fact that society depends on the ability of people to work effectively together. Students often see collaborative learning as either a potential threat to their grade, or often based on prior experiences, as an added burden due to social loafing by peers. Many students have never been explicitly taught how to manage conflict, work with others, seek understanding of alternate viewpoints, and are unaware of the data indicating that diverse groups usually reach more effective problem solutions.

Thus, we have a storm of interacting elements here of culture/consumerism, negative prior experiences (with both peers and instructors), student lack of awareness of their own level of communication and collaboration skills and how to monitor and improve collaboration, and usually cognitive development where some students on the team see collaborative assignments as about “getting the right answer” rather than being about enriching their understanding of the material and promoting critical thinking (see Nelson, 2015). Let’s use the specific example of my Abnormal Psychology course to illustrate these issues and how promoting metacognition can help.

Example of Incorporating Metacognition: An Online Abnormal Psychology Course

Students know from the syllabus that the class will involve working in teams; they also know that course objectives are weighted towards development of professional skills as well as content learning. They begin to engage with metacognitive assignments by completing two instruments1:

  1. the TTM-Learning Survey (TTM-LS) assesses the student’s degree of readiness to change how they learn and their readiness to engage with a collaborative team, and
  2. the Learning Strategies Self-Assessment (LSSA) measures how often they use known effective learning strategies and engages them with reflective questions.

These assignments are due the first two weeks, before team activities begin.

A comparison of typical responses across two particular LSSA questions reveals some helpful insights on student thinking about collaborative learning. One LSSA reflective question asks them about their personal goals for the course. Despite knowing the class will involve teamwork, only 5% of the students indicated any related personal goal for improving their teamwork skills and learning (Fall, 2018 class). However, when the LSSA asks students to review their learning strategy scores and identify their strengths and weaknesses as a learner, 39% of them acknowledged weakness in collaboration or a reluctance to work with others. For instance,

 

  • Student A wrote, “I have a hard time asking for help when I get frustrated or confused. I feel like it is my responsibility to learn the material and do not want to put someone else out by making them take the time to teach me a concept I should be able to learn on my own.”
  • Student B replied, “I noticed that I’m very good at doing things on my own but when it comes to asking for help or working in groups, I don’t do it as often.”
  • Student C said, “I have learned many strategies throughout my academic experience on working with teams and they were mostly negative.”

 

At the beginning of the semester, these instruments opened the door for students to own these emotions and experiences, to think about how they could do better, how ready they were to change, to take responsibility for their own skill development, and to at least be willing to consider the value of collaborative learning. Instructors using instruments like this have the opportunity to provide feedback, lower resistance, and engage with these students in productive ways to prepare them to do well in a collaborative course, even if it is online.

Sibley and Ostafichuk (2014) describe some interventions instructors can take to help students “buy in” to the value of Team-Based Learning (TBL) including explaining the purpose and relevance of TBL, acknowledging negative experiences, and demonstrating the difference in quiz scores by teams compared with individuals (teams do better). These efforts are useful, but they are not inherently metacognitive and could be seen as just more instructor justification. The critical task for the instructor is to foster, across the semester, metacognitive thinking and evaluation of how collaborative experiences enhance their learning and strengthen their critical thinking and communication skills. Other aspects of TBL such as self and peer evaluation, if done well, also promote metacognitive development and learning.

Making Metacognition Pervasive

To be effective, metacognitive activities in collaborative environments must occur across the semester; single assignments or events will be insufficient. For example, a week after completing the two instruments above, students complete a Personal Learning Plan. They are asked to reflect on their TTM-LS readiness to change stages and to explain their next steps to become more effective learners and team members. They also create a personal study plan for the semester.

As students launch their teams, they engage in readings about the value of professional team skills in the workplace and engage with sites like Surviving the Zombie Apocalypse (https://teamwork.umn.edu/) to identify common myths contributing to negative experiences and devise a plan for working together. A later team workshop asks them to evaluate team progress and identify areas for improvement. Online videos and class discussions on these topics, connecting the themes to professional practice in the field, are also vital. These metacognitive “boosters” help them continue progress in the development of these skills.

photo of 5 hands making a circle by gripping adjacent wrists

Wirth and Perkins (2013) note that metacognitive skills must be developed in the disciplinary context, with students questioning their own mastery, progress, and applying relevant concepts. Designing collaborative learning courses to engage students in metacognitive activities from the very beginning and then continuing that dialogue can lead to significant gains in learning content and development of metacognitive, critical thinking, and collaboration skills.

At the end of the semester, my students complete the TTM-LS and LSSA again. Some questions ask them to identify activities that enriched their learning and how they will use what they learned in the future.

  • Student A stated, “I will try to form a study group or a team to try to learn the material, because I felt the more I taught the others the more I learned for myself.”
  • Student B noted, “…Meeting in a group was probably the most beneficial way for me to learn the course material. Looking back, it’s kind of ironic that that is my favorite aspect because I fought it so hard in the beginning”, and
  • Student C, who had a difficult semester, reflected, “[My] attitudes have been changing and so [has]my way of dealing with group work. I learned I had to change my attitude in order to change the way I think about an issue…”

In total, 80% of the students made statements positive about team work and how it benefitted them in response to these questions. Building ongoing metacognitive activities into collaborative learning environments makes a significant difference to student success.

References

Kloss, R.J. (1994). A nudge is best. College Teaching, 42(4), 151-159.

Nelson, C. (2015, February 15). Fostering Metacognition: Right-Answer Focused versus Epistemologically Transgressive. ImprovewithMetacognition.com. https://www.improvewithmetacognition.com/fostering-metacognition-right-answer-focused-versus-epistemologically-transgressive/

Nuhfer, E. (2014, July 15). Metacognition for Guiding Students to Awareness of Higher-level Thinking (Part 1). ImprovewithMetacognition.com. https://www.improvewithmetacognition.com/metacognition-for-guiding-students-to-awareness-of-higher-level-thinking-part-1/

Perry, W. G., Jr. (1999). Forms of intellectual and Ethical Development in the College Years. (Reprint of the original 1968 1st ed). San Francisco: Jossey-Bass.

Sibley, J. & Ostafichuck, P. (Eds). (2014). Getting Started with Team-Based Learning. Sterling, VA: Stylus Publishing.

Stover, S. & Holland, C. (2018). Student resistance to collaborative learning. International Journal for the Scholarship of Teaching and Learning, 12(2), Article 8. https://doi.org/10.20429/ijsotl.2018.120208

Tolman, A.O. & Kremling, J. (2017). Why Students Resist Learning: A Practical Model for Understanding and Helping Students. Sterling, VA: Stylus Publishing.

Wirth, K.R. & Perkins, D. (2008). Learning to Learn. Retrieved from: http://www.macalester.edu/geology/wirth/CourseMaterials.html

1 These instruments are available under a Creative Commons license, so feel free to contact me (Anton Tolman).