293. Study Like a Champ

The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, Regan Gurung and John Dunlosky join us to discuss a new resource they have created that is designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

Show Notes

  • Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. Routledge.
  • Gurung, Regan and J. Dunlosky (2023). Study Like a Champ: The Psychology-Based Guide to “Grade A” Study Habits. American Psychological Association.
  • Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological science in the public interest, 9(3), 105-119.

Transcript

John: The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, we discuss a new resource designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Regan Gurung and John Dunlosky. Regan is a social psychologist and is an associate vice provost and Executive Director of the Center for Teaching and Learning at Oregon State University, He is the author of over 120 peer-reviewed articles and has co-authored or co-edited 15 books. John is a professor in the Department of Psychological Sciences and the Director of the Science of Learning and Education Center at Kent State University. He co-authored the first textbook on metacognition and has edited several volumes on education. Regan and John are the co-authors of Study Like a Champ, which was published earlier this year by the American Psychological Association. Welcome, John and welcome back, Regan!

Regan: Thank you.

John D.: It’s great to be here.

Rebecca: Today’s teas are:… Regan, are you drinking tea?

Regan: Just water because it is a scorcher here in Oregon. So water, and lots of it.

Rebecca: Hydration is important. How about you, John?

John D.: No tea for me for about five years, I used to really be addicted to oolong. And now I’ve gone full espresso. [LAUGHTER] Water for me today too. It’s a little bit late for espresso on the East Coast.

John: And I have an Irish Breakfast tea today.

Rebecca: Oh, John, you’re really pumping up the caffeine today. Also on the East Coast,

John: My day started at 5:30 this morning…

Rebecca: So did mine.

John: …and it’s going to be going really late. We’re in the midst of grading here, we’re recording this a little bit earlier than it will be released. So we’ve invited you here today to discuss Study like a Champ. What prompted you to create this book?

Regan: So both of us have been teaching for a long time, both of us do research on teaching and learning. And I think both of us really like taking stuff from the lab and testing it in the classroom. And, I think, as passionate teachers, we noticed that we knew the stuff, we knew what students should be doing, we told our students what they should be doing. We even tried to design our classes in that way. But clearly, we needed to do more of it. And the impetus behind this book was let’s put these expert tips in the hands of students everywhere. We didn’t want it to be just those students who had teachers like us who talked about studying, we want it to be in student hands. And that’s where it came from, specifically driven by the fact that even though there’s some really neat stuff out there on learning, very little of that, if any, is written to the student. And that’s something right from the get go when John and I first talked, we agreed that this would be in a voice that would speak directly to students.

John D.: Absolutely. In fact, I think that was the main impetus because of all the wonderful books and volumes out there on learning sciences. And just too many of them, I think, appropriately so written for teachers. And often they tend to be a little bit fact listing because of that. In other words, I’m going to tell teachers everything there is to know about learning, which can be overwhelming. And what Regan and I wanted to do is kind of find the most effective and best little snapshots of the learning sciences to share with students, the things that we think will move the needle the quickest, so to speak. So by no means do we tell students everything there is to know about the learning sciences, but we hope we tell them the best stories to get you back onto the learning track, so to speak.

Rebecca: So you begin your book by discussing a variety of widely believed learning myths. Can you talk about a couple of these myths?

John D.: Sure, one of the biggest and I really don’t think this holds true for college students, but K through 12, definitely, it’s just the myth of learning styles. So, we all have learning preferences. I certainly have preferences. I wish my instructors would sing to me all the time and do my learning through music. It’s just that’s not going to help me learn calculus, and so forth, where really hard work needs to be done. So it’s natural to accept a myth like that, because it makes everyone feel good. It really can also undermine learning because it makes us want to do things that are more aligned with our preferences, but sometimes misaligned with most effective practices. So that’s kind of one learning myth, which I hope, in 10 years, if we were to have this discussion again, it would not come up because it was completely dispelled. But there are a variety of others too, that I think can undermine student success too.

Regan: And I think something else, that we really tried to put ourselves in students’ shoes, and we built on what our students said to us. And another one I wanted to highlight in response to that question is so often I think our students think that learning is up to us, if they have a good teacher they learn. If they have a bad teacher, they won’t learn. And undoubtedly, teacher effectiveness is important, but it’s also up to the student. The student has to do something, the student has to take accountability. And early on in chapter one, we show that at least half of the variance half of the accountability for learning is in the student hands. Teachers are also responsible, but really, if a student isn’t doing their bit, then learning is not going to happen. And now once you are ready to do your bit, well the rest of the book tells you what to do. But you’ve got to realize that don’t just sit back and go “Come on teach. Do your magic.” We all like doing our magic. But we can do all the magic in the world if you are not joining us in this adventure, and I’m thinking adventure a lot, because Zelda Stairs of the Kingdom just got released recently, and my household seems to be quite into it. So yeah, join us on the adventure.

John D.: Absolutely. Let me just jump in with one other learning myth that really gets me, and it literally gets me, because I fall prey to the same thing. That’s if you’re struggling, you’re not learning because it’s frustrating. And I often try to pick up new skills, I’m teaching myself to play the guitar now. And I struggle, and I get frustrated, I want to throw my guitar down because things aren’t happening. And I have to remind myself that sometimes it’s the struggling when you’re doing the most learning. And if we could allow students to realize that if you’re not struggling, you actually may not be learning, and that you should embrace the challenges, the hurdles that are in front of you. And it turns out, some of those hurdles are embracing the most effective learning techniques, which produce struggles in learning, but yet also produce the best long-term outcomes. Whereas other ways to prepare for classes and so forth, the way I’d like to teach myself to play guitar, makes it seem very easy. But then there’s really no progress being made. And if there’s any method I could really want to undermine, that would be a major one, that sometimes struggling is a good thing, not all the time, and it’s up to the students and the instructors to figure out is this a good struggle or bad struggle and how to correct that.

John: Those times when we try to give students desirable difficulties, they often find those to be somewhat undesirable. And I often get comments in my course evaluations that “he’s making us learn it ourselves.” And my response is always “Well, I can’t really learn it for you, I can give you the tools, I can give you resources, I can help you learn this material, but ultimately, you have to do the learning.” Going back just a little bit to your comment, John, about learning styles, I had a discussion in an online class, which is mostly upper-level students. And about three quarters of the students in my class were very strongly convinced of that myth of learning styles. So it hasn’t gone away. We hope it ma, but it is very pervasive, certainly in K through 12, where they’re regularly testing it, but some of the students have mentioned that they’ve been tested here at the college in some of their classes. And it’s again, really frustrating to see that, but it’s pretty pervasive.

Rebecca: Yeah, I was struck too, John, when you were talking about both learning styles and the struggle, because as I was working with mostly graduate students this semester, I had those exact conversations with those students. But it may have been the first time that they’ve really come up with some barriers that they faced in learning, and really had to reassess how they were working towards their goals.

John D.: Absolutely, and nothing but utmost compassion for students and sympathy, because in many ways, the experiences they had before college allowed them to use ineffective strategies and still get by. So things didn’t seem like a struggle, the difficulty now, the first time, then you hit a struggle, when you really have to change strategies or techniques, you may not realize what to do. It might seem frustrating and overwhelming, when in fact, now’s the time when the real learning begins.

Rebecca: Yeah, and I think sometimes when the subject matters new, as well as the strategies are new, students can immediately feel a little cognitive overload. [LAUGHTER]

JOHN D. Absolutely.

John: And part of that I think may be related to how they’re often taught throughout their whole career. In general, much of the advice they get in elementary and secondary schools, and sometimes in college, encourages them to adopt strategies that may not be that effective. Often, people still use high-stakes exams. And people don’t do a lot of interleaved practice or spaced practice, which are things you talk about really nicely in your book. And the incentive structure for students is perhaps to favor things that align well with short-term recall without that longer term learning. And your book can help, if we can get students doing this, but it seems to me like we also have to do a bit more work and getting faculty to adopt some techniques that may encourage students to use techniques that are more effective. Might this be something we could build into our classes a little bit?

Regan: Actually the riff over a couple of different things you said, John, I think there are two things going on here. One is, a lot of the issues that we run into with our students are actually pretty easy to pinpoint. And that is high school. And when I say high school, I’m not saying “Oh, high school teachers,” I’m saying the whole high school environment. One of my favorite examples is the syllabus. Many of us college and university faculty complain that students don’t read the syllabus, and I had a great conversation with my 17 year old. I was talking about the syllabus and he said, “Yeah, the syllabus is useless.” I said, “Excuse me?” and he said, “Here’s what we do with the syllabus. We spend a whole day, a whole day, reading through every word on the syllabus, but the syllabus is not really designed to help us learn.” And I looked at his syllabus. And sure enough, the syllabi he had didn’t really map on to the best practices that many of us listening employ. And I could completely see why he had that whole. “Yeah, I’m not going to read the syllabus when I get to college. I said, “And of course, you will now,” but it was a great conversation where I really got to see the mindset. And I think that’s just one example, where a lot of this happened, where it’s like, let’s see, what have they just been used to? Then add on top of that, how learning changed during remote learning. That’s another whole mix to the whole issue. And then, add one more, the misinterpretation of, I think, learning styles. Before we started this call, we were joking about the fact that our book is now in an audio version. Now, because it’s an audio version, does it mean the auditory learners will be better than those….? No, it just means that sometimes when you want to listen to something that works well, other times reading something works better. So it’s not this whole “Oh, look, there’s this one modality and I’m a person of this one modality.” And I think that preference that John D. so nicely used… it’s preference, right? …and we have preferences. But let’s also not forget the take home message of that Pashler et. al 2008 study, which is the more styles that you use…. And here’s where it goes back to desirable difficulties, mixing it up, is actually better for learning, no matter what your preference may be.

Rebecca: Oh, but mixing it up is so uncomfortable. [LAUGHTER]

John D.: It makes things a lot more difficult, right? One advice I would give too, that my teaching has changed dramatically, as I’ve gotten more into learning sciences. And at least for someone who teaches, I hate to say this, a non-essential course course that I love and courses that I love for undergrads, I realized I was teaching them too much. So I put them into almost a defensive mode, where the only way to prepare, because they were overwhelmed, is to cram as much as they could and hope to pass the exam because it was a tsunami of information. So over the years, I’ve followed the model, less is more, it’s up to me as the instructor to figure out what is really essential, and how to give them just as little as possible to make their experience as large as possible. So that they then can use the techniques to learn the content well enough so that they can keep that content with them a lot longer period of time. So it’s really kind of changing styles. Now, don’t get me wrong, if I’m teaching chemistry to pre meds, and you need to know chemistry, then you need a different approach, because there’s a lot to learn, it’s difficult, and aspects of our book really focus on how to meet those challenges too. And I think students, just like instructors, need to pick and choose where they want their battles, you don’t have to ace everything as a student, but decide what you really want to do well at, and then use the most effective practices to nail it as you work forward toward a long career and lifelong learning.

John: One of the things you include early in your book is a list of what research finds to be effective and what research finds to be ineffective, side by side, which for faculty use could be really helpful except for some degree of similarity between those two lists. Could you just talk a little bit about that.

John D.: I like to think about it, as much as I’ve not talked about it like this, as more or less effective versus effective and ineffective. And I’m not gonna like reference to that table necessarily, but let me talk about one that I badmouth a lot, which is highlighting as a learning strategy. And like Sharpie has not contacted me with a lawsuit or something because it sounds like I don’t want people to have highlighters. And what we mean by less effective as a learning strategy here is, is that using a highlighter is just the beginning of a learning adventure and not the end of it. So there are great uses of a highlighter of rereading material, of these things that really don’t lead to a great deal of learning, because they kind of are the stepping off point or the catapult for learning, so to speak. So is using a highlighter ineffective for learning? Absolutely, it doesn’t help you learn the content that well. But what I recommend to students: to highlight everything they want to learn as kind of a different approach to that strategy, so that they then apply the most effective strategies in learning it. Yes. So as much as we kind of play off ineffective to effective, it’s more like every strategy in its right place. And some strategies that can be generally ineffective for very specific uses might be relatively good, and what I think Regan and I try to do, because it’s way too much to think about if you’re not a learning scientist, like one exactly show us the specialty strategy, but we try to focus on are just strategies that students could learn just more generally to do well in any particular content. So kind of low hanging fruit, so to speak, where you don’t have to be a learning scientist to make decisions about which specialty strategy to use in every place. But here’s some kind of fail-proof strategies that can help you anywhere.

Regan: I want to actually say there’s that flipside too and John, this is something you asked about earlier when you talked about teachers changing what they do, and I think John D, and I joke about the companion volume to this is Teach Like a Champion. But I think the nice thing about that table of high utility and low utility strategies is that it’s not just the student who needs to be aware of the fact that some are high and low. But I think we take it further to say why are some of those high and low, and then here’s the key companion piece, which is, we hope that instructors reading this can also take those tips, and be ready to share different options with the students. Because I know both John and I have compared notes on this, when a student comes to our office after an exam who’s not thrilled with their score, who has not scored well, we both do some pretty intensive. “Alright, so tell me exactly what you do.” Just that conversation is so important. We both ask to see their notes. And of course, every once in a while a student says I don’t take notes. Well, let’s start right there. And that’s why we have a whole chapter on note taking, and why take notes and how to take notes. But I think that’s exactly it, is so many of us. and I’ll say us, not just our students, so many of us are so used to doing things just one way that we’re not ready. And we don’t look enough about is this really working or not. And I think that’s what we really push with a lot of very real examples ripped from the headlines of our lives, as it were, where we talk about, look, this is what somebody’s tried, and here’s what they did wrong. And here’s how they can make it better.

Rebecca: I wanted to talk a little bit about that chapter on notetaking. Faculty often make the observation that students don’t take effective notes, but then don’t necessarily respond by teaching students how to take notes or giving them some strategies to improve that skill. Can you talk a little bit about some of the things that finally might show students or offer students to take more effective notes?

John D.: Absolutely. So the one thing that I would recommend highly is that if we all agree that taking notes in its own right is not where the learning occurs. And in fact, research comparing people who take notes versus who don’t take notes typically showed no differences between the two. Now, the key is, when you look at these studies carefully, you’ll realize the folks who took the notes, and who didn’t take notes do equally poorly. So it’s not like they do equally well. So the notes are a artifact that in many cases students can use to really learn the content well. So the question is, how do you get all the correct and appropriate information into those notes? So one thing that teachers can do, that research shows really helps well, it’s just flag important content, literally tell students “Okay, time to take notes, this is critical, I need you to know this.” One of my favorite statements, ”The next thing I say, will be on the exam.” [LAUGHTER] And see how many people start taking notes. Now, of course, I would love to see notes that are embellished with examples, with questions, and so forth. But at least getting the rudimentary content down there as an artifact, so students have it to utilize as they’re preparing is so important, or having teams of students take notes and share those things as well is also good. So I think for teachers to be compassionate, but also to help your students sometimes identify what’s most important, by speaking louder by saying, “Hey, this is really critical.” Sometimes I have to even flag it to not only just critical, but it’s going to be an exam, you’d think those two things meant the same thing, but sometimes students need a little help. I think it’s important, and there’s nothing wrong with telegraphing what is most important in your class. In fact, let me restate that. There’s everything not right about not telegraphing. Now, try to parse that. But what I’m saying is, we should be telling the students what they need to learn, how we’re going to test them on it, what we’re going to test on, and allow them to meet those successes. And I think good note taking, and helping students take notes by telegraphing what should be in there, can be very useful, in my own experience, and at least some of the research too.

Regan: Notice what else is in there. If we do want students to take good notes, we don’t want to be talking extremely fast. We want to make sure we have pauses, we want to make sure our slides, to the extent we use them, are not so packed with information that either they want to copy it all down or they get it all as a handout so they don’t have to, but there are all these nuances. And I want to go right back to that finding about the students who take notes and the students who don’t take notes. The bigger issue is what are you doing with those notes? I think far often students just think about it as “I’m going to record what the instructor said.” Well, you’ve got to go back and revise them. You’ve got to go back and check and see if your understanding is right. You need to review them. The issue is, I think, when people take notes, they’re not doing enough with those notes. And there are ways where we can talk about how to take better notes and they’re not. When I teach 100, or intro level classes, every once in a while, I will stop, and I do this during the first week, I will stop. And I’ll say, “Alright, here’s a reality check. If you took good notes, you should have the following. How many of you have it in your notes?” And I do that a couple of different times that first week or the first two weeks to get into that habit of you should be taking notes, you should be doing something about it. I also want to say something that didn’t make it into that chapter on note taking, but it’s gone up another shot, because I’ve seen this so often, in the last year and a half or two years, students taking photographs of the slides. Reality check, people, that’s not taking notes, taking a photograph of a slide is not taking notes. But I think in some students’ minds they’ve taken notes, just more effectively. And there are studies now rolling out that show that the photograph slides versus note taking versus controlled no notes. that the photographs and no notetaking is doing the worst on quizzes. But I think that’s something else we’ve got to take up” how you take notes is important and taking a photograph is not it.

John D.: Let me spin back to something that Regan said that I think is great, too, as far as helping students develop better notes, I begin, and I know Regan does this as well, every class with a no-stakes quiz. And I use that for a variety of different reasons, some of which we talk about in our book, but one of them is first try to answer the quiz multiple choice question without the alternatives from memory. And if you got it from memory, that’s important, because that’s going to help you learn. And then if you don’t get it from memory, now see if you can answer using your notes from the last class. And if they can’t answer the question using their notes, we have issues. So they’re not taking complete notes, they don’t understand their notes, or the issue could be with me. Occasionally, I look at the notes and realize, “Oh, my goodness, I must have said something wrong last class, because everybody’s notes are incorrect.” [LAUGHTER] And if everyone’s notes are incorrect, the source of the error is probably me. So it’s a way to help students understand that the notes are the vehicle to understand the content that we really want them to learn. And there’s ways to reinforce that through no-stakes quizzes, and so forth to help them understand that what we’re doing in the classroom is providing all the scaffolding to help them succeed.

John: One of the things you emphasize early in the book, and throughout the book is the importance of students developing their metacognition as well as their self-regulation skills. Could you talk a little bit about why that’s important, because I think that’s something that’s not emphasized enough throughout their educational experience,

John D.: I’m gonna just focus on one aspect of our particular viewpoint on self regulation, and then I’ll let Regan go with the more fun stuff. I want students to really succeed in school and in college. But quite frankly, as a college educator, I want students to succeed in life after they leave here. And one of the most important skills that anyone can learn is time management, that’s about self regulation. And believe it or not, I have actually students tell me, it’s like, “Well, I know, managing my time and developing plans, that’s just not my cup of tea.” (speaking of teaching for tea). But it absolutely should be, because effective people develop goals, plan on how to get them, and manage their time to get there. So if students can take something from this book that I think is highly valuable about self regulation is to look at our examples and our encouragement and inspiration to manage your time, make goals and plans, and develop a time management strategy that works well for you. Because that’s ultimately going to lay the foundations for a successful use of other strategies we discuss in the book, but also for a highly successful life, whether it be success in your job, your hobbies, raising a family, or whatever, successful people manage their time and self regulate. That’s something that we really do push in the book, and some of it’s not necessarily metacognition, per se, but it’s about really taking ownership of your life and deciding how to get things done effectively. So time management, for me is so important aspect of this book.

Regan: I think whenever you think about self regulation, I really try to stress two elements of it on one hand, especially think about metacognition is the: “What do you know? Do you know this stuff?” And there’s that whole “Am I ready for this exam? Do I really understand this concept? Do I know how to do this?” That’s the classic metacognition, but even more broadly, when we talk about plan and monitor and assess your knowledge, that’s, I think, really important as well. And of course, the tips such as retrieval practice is great for the assessment too and to assess yourself. But I think I love the way John talks about time management because when we talk about self regulation, a big issue that I hear from students a lot is where those two things interact, not being able to regulate how they use their time, especially when it comes to social media. And students will say to me, “Yep, I start scrolling Tik Tok, and before I know it’s an hour later. Well, that’s a basic self-regulation issue. That’s a time management issue. That’s a planning issue. And I think in our planning chapter, we recognize this and we tell students “Look, it’s okay to enjoy social media. But guess what? Plan your social media timr. So it’s not like you’re saying, I’m not going to use social media. Be real. If that’s what you enjoy, sure, look at some social media, but plan it out, allocate time for your social media, just like you allocate time for your work so that everything fits into your schedule.”

Rebecca: Allocating time, it feels like have a lot of conversations with students about that on a regular basis.

John D.: Oh yeah.

John: Are there other topics you’d like to emphasize about your book that we haven’t touched on yet?

Regan: Well, I think one thing that if you’ll notice, the last chapter in our book is all about things that, for the longest time, nobody thought related to studying: sleeping, eating, physical activity. But I think that is just so important. That is so important. And gosh, I know, I don’t think I ever shared this with John. But sometimes I almost played with putting that chapter first. We did put it at the end, because I think it’s really important. But I think most students don’t realize how those life things interact. And I think for those of us who read up on higher education, there’s just so much written about burnout among students, among faculty, and all of that relates to those things we talk about in the last chapter. It’s prioritizing sleep and prioritizing good eating and prioritizing physical activity, prioritizing just getting out and getting some air. And some of my favorite comments that I get back from students… it happens a lot when I teach health psych… but even otherwise, where they will say, “you know, yeah, I caught myself scrolling, I went outside for a walk, and it was just five minutes, but I came back and I could tackle things better.” And I think, people, it’s 360 degrees of living is what we’ve got to practice.

John D.: And I love that Regan. Because obviously, we saved the best for last with that chapter. But, come on, to do all these things. It’s a time-management issue. If I’m going to work, all of this in, how can I do so so I have enough time for play and I have enough time for work, which are two important aspects of life. So, totally agree.

Rebecca: Well, they always wrap up by asking: what’s next? [LAUGHTER]

John D.: Wow, that’s a big question. And what’s next, I would say for me, the strategies that I think are most effective, that work the best, unfortunately, have also been investigated the least, because they’re difficult to investigate, they involve multiple sessions for students to engage the same material, and so forth. So as someone who was kind of born and raised in a laboratory, either my own lab or laboratory of the classroom, what I’d love to see is just much more evidence-based research focused on further understanding, not only what works best, but how students can engage in the best practice in the most efficient way. So it’s balancing both an understanding of wanting students to obtain their learning objectives, but also understanding that they want to do so effectively and efficiently. Because there’s so many aspects to life beyond just school. So I’m always going to say more research, although I could totally understand some would say more application too, but I’m gonna go what’s next is some more research and all these really effective techniques and how to use them more effectively.

Regan: And it’s that last part, John, that is what’s exciting me right now, I’ve been learning a lot more about the whole field of implementation science, which is just because something is effective, and something is efficacious, what are the factors that influence its implementation? …because whether it’s a student reading our book and trying to implement our recommendations, or a faculty member, instructor, teacher listening to us and trying to implement what we’re saying, there are still nuances, there are still contextual factors. And I think we’re just getting better at intentionally and systematically teasing apart what some of those issues with implementation are. And for me, that’s been pretty exciting. I was fortunate to read a lot of a fellow Oregon State University, new PhD, Dr. Rachel Schweitzer, who’s done a lot of this work on implementation science, and some of her ideas. I’d love to be able to test them in my class, to dovetail with continuing research on study techniques and how to make it happen more,

Rebecca: Some really important stuff that needs to happen, for sure.

John: And thank you for this book and all the other research and work that you’ve done in support of improving student learning.

John D.: Thanks for having us on.

Regan: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

289. The Cognition-Motivation Connection

Emotions can have both positive and negative impacts on learning. In this episode, Michelle Miller joins us to explore the relationships that exist between emotions and learning.

Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University.  She is the author of Minds Online: Teaching Effectively with Technology and Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World. Michelle is also a co-editor, with James Lang, of the superb West Virginia University Press series on teaching and learning.

Show Notes

  • Miller, Michelle (2023). “Revisiting the cognition-motivation connection: What the latest research says about engaging students in the work of learning.”  March 3.
  • Miller, Michelle (2022). “Ungrading Light: 4 Simple Ways to Ease the Spotlight off Points.” The Chronicle of Higher Education. August 2.
  • Remind
  • Transparency in Teaching and Learning (TILT)
  • Sathy, V., & Hogan, K. A. (2022). Inclusive teaching: Strategies for promoting equity in the college classroom. West Virginia University Press.
  • Ariely, D., & Wertenbroch, K. (2002). Procrastination, Deadlines, and Performance: Self-Control by Precommitment. Psychological Science, 13(3), 219–224.
  • Abel, M., & Bäuml, K. H. T. (2020). Would you like to learn more? Retrieval practice plus feedback can increase motivation to keep on studying. Cognition, 201, 104316.
  • McDaniel, M. A., & Einstein, G. O. (2020). Training learning strategies to promote self-regulation and transfer: The knowledge, belief, commitment, and planning framework. Perspectives on Psychological Science, 15(6), 1363-1381.
  • Miller, Michelle (2019). Attention Matters. Tea for Teaching Podcast. Episode 86. June 19.
  • Michelle Miller’s R3 Newsletter
  • Sarah Rose Cavanagh’s Once More, With Feeling substack
  • Cavanagh, S. R. (2016). The Spark of Learning: Energizing the college classroom with the science of emotion. West Virginia University Press.
  • Cavanagh, S. R. (2023). Mind over Monsters: Supporting Youth Mental Health with Compassionate Challenge. Beacon Press.

Transcript

John: Emotions can have both positive and negative impacts on learning. In this episode, we explore the relationships that exist between emotions and learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology and Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World. Michelle is also a co-editor, with James Lang, of the superb West Virginia University Press series on teaching and learning. Welcome back, Michelle.

Michelle: Oh, thanks for having me. It’s great to be here today.

Rebecca: Today’s teas are:… Michelle, do you have some tea?

Michelle: Well, not exactly. I’ve started hydrating with fruity water today. So, I’ve got my water jug and I’m working on it.

John: And I have just a little bit of a peppermint-spearmint tea blend here. And the reason is just a little bit as this is our third podcast of the day today. So I didn’t have a chance in between them to go back to my office and get some new tea or some new hot water. So I do have a little bit to get us started here.

Rebecca: A tiny bit left from my pot of blue sapphire tea.

John: …which is much more colorful.

Rebecca: It is, but not in my cup, only in the pot.

John: We’ve invited you here to discuss your March 3 blog post, addressing the relationship between cognition and emotion. In general, how is cognition influenced by emotion,

Michelle: I’ve been interested in this connection for a while and watching the evolution from within my field of cognitive psychology and kind of moving away from the approach that I came up with when I was just starting out as a graduate student, which is I recall was this kind of oil and water conception of cognition and emotion that here on the one side, we’ve got thought processes, we’ve got memory, and so on. And on the other side [LAUGHTER], we’ve got the emotions and so on. And we’re just going to really work from in our subfield to try to get our arms around just these cognitive processes, and don’t worry about the rest of it. And now, I think that most cognitive psychology theorists in the field would say that, yeah, our cognitive processes are definitely shaped by and infused by what’s going on kind of over in the emotional processing systems of our mind and in our brain. And if I had to describe, just from my own perspective, what I see is a change over time and an evolution in our field, we’ve kind of gone from really talking about parts of the mind in this very compartmentalized modular way, where different parts do different things pretty much independently. And now you see more discussion of how these different parts have interplay with each other, how they give what I would think of as a sort of a soft input to other subsystems, or even set some constraints on what those other systems are doing without totally determining them. So I think we are moving into this more nuanced view of how those two things work together. So that, yeah, our emotions affect what we believe, they also serve as a way to almost elevate or suppress different aspects of what we’re processing so we might remember things in a particular way, or think about them in a particular way. And it’s neat to me too, as somebody in the field, because I look and I see clinical psychologists, the people in the area of psychology who work on how do we help people in therapy and help people with different disorders and challenges. They’ve known this for quite some time, but they’ve looked at it sort of in reverse. So if you’ve ever heard the school of thought known as cognitive therapy, or cognitive behavioral therapy, one of the core tenets of that approach is that the emotional side of how we function, our emotions, are affected by our cognition. So what we feel, even our mood states and so on, that’s fundamentally driven by things like what we believe. And so they’ve come up with these really exciting and powerful techniques for addressing beliefs that people have and thereby affecting their emotions. So we can take a cue from that and have this more nuanced view of the interplay. So back to cognitive psychology. I also come at this really philosophically as what we would call a functionalist, [LAUGHTER] that’s sort of a lens through which I see how we address questions in psychology. So when we say, “Well, why does the mind work in this particular way? Why does it have this component or why does it do this in this way?” I would look at and say, “Well, how does that help us survive? What’s the function for helping us really survive and thrive in our world?” And when we look at things like emotions, our emotions are there for very functional reasons. I believe our emotions exist in order to kind of move us towards things that help us in our survival and move us away from things that are going to be a threat to our survival. And also they serve in this way to kind of alert us to what’s relevant. So it’s almost like a relevance mechanism. So if something provokes an emotional response in us, that may be an old shortcut that our mind has to say, “Yes, this is something that’s important. This is something that maybe you want to remember and that you want to pay attention to.” So I see emotions as kind of a feel for relevance, and that’s something I’m sure we’ll get into in our conversation about teaching and learning. later. And all of this is a practical issue too. I tell the story sometimes about Minds Online, and writing that book, where I got to about midway through the book and literally had this crisis, I remembered it happening like in the middle of the night, there’s something huge that’s missing right around this point in the book. And that book, for those who have taken a look at it, it takes a very cognitive view of how we select and use technologies. But I came to this realization, we can’t really talk about how to maximize the effectiveness of those approaches, unless we also talk about why students are going to do them in the first place, and how we can get them motivated to do them. So, in that book, I ended up covering some very basic elementary foundational concepts in motivation and motivation theory with that idea of what are just the essentials that every teacher needs to know and how might that also get involved in how do we choose certain technologies? How do we set things up in a particular way, for example, in an online course to keep students moving and that keep them putting in that productive effort. And so that’s been around in the back of my mind for quite some time. But now I’m reading all these new articles and this wave of interesting new research that is finding yet more connections between those two sides of the mind. And so how to get students to engage in strategies that work from a cognitive perspective and how to direct that feel for relevance. And early on in my career, as I mentioned in that blog post, I look back and it seems so harsh now, like, well, how do we get students to be accountable. And now I’ve kind of shifted that along with many others towards really looking more at the support side of this and bringing in things like empathy for our students, I don’t think I’ve ever been one of those super punitive “look to your left, look to your right” kinds of teachers, nor have I ever really advised that to their faculty. But I’m realizing that in this really critical case that I’m looking at this relationship in new ways, and I’m excited to share that.

Rebecca: So there’s been a lot of discussion about student motivation and engagement…, a crisis in it. [LAUGHTER] A lot of faculty have reported students being less engaged or less motivated. How can we, as faculty, address some of the challenges that people are experiencing at this moment?

Michelle: And it is such a pressing question, and that’s another thing that’s just really been registering as I’ve had my antenna up about what are people talking about right now? What are they bringing into conferences, and so on? And first off, as a little bit of a skeptic, I have to say, “Well, I think that we still need some more information to nail down exactly what the extent and the nature of the engagement crisis is.” And I think all three of us are attuned to what I guess you can call the fallacy of “students these days.” [LAUGHTER] So as so many people have observed, it’s so tempting to have that filter on of like, “Well, back when I was a student, I was always intrinsically engaged in my classes, I didn’t miss assignments, and so on and there’s a downward trend.” So there’s something about that. I put on my skeptic filter when I see that. But that said, we do have these experiences. And I don’t think regardless, even if we look back and say, “Well, maybe this wasn’t really part of a bigger trend as we thought.” Even if that were to happen, are we gonna look back and say, “Well, we shouldn’t have worried so much about engaging our students, we can almost always stand to engage them more.” So with that big caveat, I think that we should also be really reflecting on and separating out, as much as it’s really possible to do so, disengagement from other related things like prioritizing. I don’t have the capacity to cover all that I need to as a student, and perhaps also as a family member, a parent, a worker, and so on. So here’s how I’m going to go at it, or even just straight up overwhelm, and I think we can look at that from our own perspective, too, and say, “Well, right.” I think we’ve also seen quite a few faculty professional development directors and others who work with other faculty to say, “Oh my gosh, I put up a half a dozen workshops, and I’m having trouble filling them. So we too, as a lot of our demands have converged over the last couple of years, and as we’ve coped with those stresses, we too. It’s not that we’re disengaged from what we’re doing, but we’re having to make some different choices out of necessity. We have the economic costs of college and that whole dynamic that’s going on as well. I’m no expert in that. But I think we all know that students today are working more jobs, succeeding at every single course and getting through as quickly as possible is an economic necessity and so on. So the stakes are very, very high for students, and students are dealing with that. And so that’s one also very important thing to think about when we’re looking at this. So with that, though, have students been more disengaged? I mean, my experience immediately coming back to in-person teaching, I found myself that students were really excited. At the risk of sounding very strange here, it was like a box of excited puppies: Oh my goodness, we’re all here in the classroom together. And I felt the same way in some ways too. But really directing that in some, again, productive ways is what we have to do as the leaders of our classes. Now to practical tips for what can we do. If there’s a disengagement, students are elsewhere, they’re not doing the work, or they’re all excited but they don’t know how to manage that. But here’s a couple things that I think are very practical. So I’m a big advocate these days for flexibility and approachability in what we do. So I wrote a piece last year titled, “Ungrading Light…” I think that was the catchphrase in the title… which talks about “Okay, without sort of throwing out grades and say, ‘Well, students have the wrong motivation when it’s all about the grades.’” If we’re still going to have grades, what are some positive ways to keep students really focused on the learning and engaged with the learning and not just like, checkpoint, checkpoint, how do I get through this? And I do think that even some basic changes to policy can help here. So things like I really have gotten very flexible on deadlines. The caution here that this is going to look very different for people with different course sizes, section sizes, different disciplines, what the learning objectives really are in your course. So I don’t want to imply that everybody just can do this in the same way. And as I also mentioned in the piece, things like very flexible deadline policies can present a professional risk for people who do not have the security of, for example, tenure, and people who are historically minoritized, and are going to elicit different kinds of reactions from students to play out on things like end-of-semester evaluations. So for example, faculty of color. So with those big cautions in mind, now, here’s been my experience is I communicate with students… I say, “I want to be approachable,” I want to really show them and not just tell them that if you come to me and say, “I was pulling double shifts all weekend, and I need to do this paper draft, and I know that, but I need another two days,” that I’m not going to come down on them in a harder, personal way. And if they do that, just not all the time, I will say, “Yeah,” and then my catchphrase right now is, “Take the time, you need to do your best work.” And that turns out actually to be really good for my motivation, too, because I would really rather read what they put together [LAUGHTER] with a little bit of more time, that’s all about I want to do something I can be proud of in this course, and actually walk away with great knowledge. It’s more geared towards that and less geared towards “Oh my gosh, didn’t come in until 11:58 when it’s due at midnight, and I sort of just checked that box again. So that is something as well and other ways to be approachable, that can also elicit more engagement. How do we know students are engaged? Well, when they reach out to us. And so, here again, different individuals have to decide their appropriate comfort level and parameters. But I have a syllabus statement that says here are all the different ways to get in touch with me. If you’ve got a long question, and we need to talk, I’ve got a scheduling program, you click a button, and boom, now you have access to my calendar, and you can get on my calendar the same way my colleagues can. And that’s good. If you have a really quick question like, “Oh my gosh, there’s one thing I need to do in order to finish this assignment and actually be successful, then you can text me or send me a message in a program such as Remind, which can kind of buffer so we’re not trading phone numbers. But, that immediacy, it has not really resulted in this giant pile up of lots of inappropriate communications, which is what I was always warned about when I was coming up as a teacher. But instead, students get the question answered and then they can kind of stay engaged with the flow of what they’re doing. So just basic ideas, but ones that I think can help move us back towards a more engaged setting where students are excited to be there and so am I. If I could add one other thing here, too, we can also take a page out of the transparency philosophy. So if you’re familiar with Transparency in Teaching and Learning, the TILT framework, it’s so powerful. And it’s all about giving more explicit directions to students, as well. What you may read as disengagement or not caring, might be “I don’t know where to start and now, I really am feeling either alienated, overwhelmed, or something in between.” And I think that’s another we can all relate to is, we’re a lot more likely to take the first step down the path and keep going if that first step is lit up, or maybe if the whole path is lit up. So taking that little bit of extra time to say “And here’s where to start and if you get stuck, here’s what to do.” That can also help.

John: I’ve been observing the same sort of issue that many people have reported of students not completing work. I’ve seen students being much more excited to be back in the classroom, and they tend to be fairly engaged in classroom activity. But what I’ve been seeing and what a lot of faculty have been saying is that students aren’t doing the work outside of class at the same rates that they used to do. And one of the concerns in terms of making your classes more flexible, in some cases, you can do that really well and I do allow that with many of the assignments. But in classes where the material builds from week to week, if students start getting behind early in the semester, they’re going to be struggling a lot more later. So I have different policies depending on whether they’re producing something, some type of educational project… a podcast or something similar… as they do in some of my classes, then they can have more time. And I give them as much time as they need to do that with multiple iterations. But with other things like reading the materials online, where there’s some embedded questions, and so forth, there, I do insist that they get it done by a certain time, because then when they come into class and they’re asked clicker questions, some of which they’re graded on, they’re not going to be successful in that if they haven’t done the basic reading. And that’s where I’m seeing a lack of engagement, outside of class. I’ve had many fewer students complete the readings before class, or even weeks after they were due, they’re still not completing some of those readings. And that’s the concern that I’m having. And I have to say that I’ve also observed some of this with faculty too, that attendance at professional development workshops have been a lot lower this year than in the last couple of years. And some of it may be because of burnout after the pandemic. And I should note that on my campus, we’re also transitioning to a new learning management system. And a lot of people have been struggling with that, which takes up a lot of their time, reducing the amount of time they have to learn other new things while they’re struggling to learn the new system. But this issue of engagement does seem to be impacting the amount of learning that I’m seeing, at least in my large intro classes, I’m not seeing it so much in my upper-level classes. But I’m wondering if some of this may be because we have students who’ve spent a year or two with remote learning in schools that often had very few resources to do that, well and students may have just gotten out of practice with doing a lot of work, because in many school systems, students were just passed on to the next grade level without necessarily learning very much in many classes.

Michelle: Yeah, I’ve seen this dynamic, actually even at my upper levels as well became rather glaring the first time that we went back to an in person symposium, it was the kind of capstone experience in this class was to bring some research to the symposium. It’s a wonderful experience, but it dawned on me partway through the semester to step back and just say, “Okay, how many of you have never done a presentation like this before?” And yeah, previously, most of the class would have had some experience either in an in-person research lab that they were in, or in a methods class or something like that. It was one of those head slapping moments, at least for me, feeling, “Of course, of course they don’t know.” And I try to come at it like, “Well, this is the time to do what I probably should have always done for what was previously a small group of students. But it’s still an important group of students who are sitting in the back going, ‘Oh, my gosh, I feel lost. I don’t want to even raise my hand. I don’t know what she’s talking about with the poster or participation or even things like what to wear.’” And so I did, I went back and dusted off and created a few stopgap materials. I found some things out on the web that actually demonstrated a poster presentation that was in progress and what to do and not to do. So it can be an opportunity to do more of that transparency and kind of scaffolding and bringing everybody up. But yeah, it can be shocking to stand back and say, “Okay, who has not actually done this thing that I kind of always assumed would be the case by this level,” regardless of what that is.

Rebecca: I’ve experienced this, even with graduate students, this lack of knowledge of certain kinds of academic experiences, in part because they were learning online or not doing things in person, and now they’re in in-person classes and having any in-person experiences. So I had the same experience, Michelle, but with graduate students, and needing to really build in some transparency there that maybe didn’t need to be there before because in their undergraduate experience, they were very likely to have had a similar experience.

John: And as you said, Michelle, giving students more structure and more support is something that we probably should be doing anyway. We just finished a reading group on Viji Sathy and Kelly Hogan’s book on inclusive teaching. And that’s the message at the heart of that book: that giving students support will help all students at least some and will especially help those students who come from backgrounds that provide less preparation for success in college. So to the extent to which we as faculty all learn that lesson, that giving students more support is useful, it’ll be a better environment.

Michelle: I agree and what seems implicit in that., how you’ve put that too is, instead of like, “Oh my gosh, another thing I have to cram into the semester” …for our motivation and our engagement is to say, this is part of one of the most noble pursuits that we can have as educators, to give it that meaningful frame. So yes, a hearty I agree with that book, in particular, and their framework. And for me, that helps me kind of say, like, “Okay, yeah, this is not just an extra add on, this is what we’re here for.” And if I’m trimming back a few extra articles, or chapters, and I have done that, to some extent, in favor of being able to go more deep and into content, and be more supportive in these positive ways, I think that’s a win.

Rebecca: I think one of the things that I’ve noticed or experienced recently with students is high engagement in class, high engagement in the subject matter, but we’ve had really interesting conversations about procrastination or not doing things outside of class, largely due to a lack of confidence, or striving for perfection that doesn’t exist. And there’s a lot of that emotion around that. And so a lot of my students have talked about that, or shared that with me, which I’m grateful that they’ve shared that with me. But that’s what’s preventing them from getting started.

Michelle: Absolutely. And I think folks who follow the research on procrastination out there, it’s not as much in my specialty area, but I do think it’s fascinating. And it’s another one of these touch points between what we believe, what we feel, what we’re motivated to do, and then in turn, what we remember and what we learn. So I think that for people who are interested in this whole topic of procrastination, why does it happen? What are some really good ways to talk about it, and address it, there’s new stuff coming out. And it’s a good thing to talk about with students as well, I think years back, it was almost a taboo subject. But now from what I hear you saying, you just bring it up with students, and we can all talk about it not as like, “Oh, that’s some terrible thing that other people, bad students, are doing. This is all of us, right? [LAUGHTER] We live in a world of abundance, but also abundant distractions, and so many things competing for our time. So I like this idea of opening up that line of communication, saying, “What do we all do to tackle this when it occurs?”

John: Dan Ariely had a paper a number of years ago, where he did an experiment in class, I think it was an economics class, actually, where students wrote papers. And he and some co authors had two sections of the class, where in one section, students had three papers with fixed due dates spread evenly through the semester. And in the other section, students were able to pick their own due dates. And there was a penalty in either case of one percentage point a day for each day the work was submitted late. And what was found in that study is that the students did best who either had fixed due dates, they had higher quality papers, and higher quality work, and so forth, they wrote more, and the quality of the work was much better when they had fixed due dates, or when they chose evenly spaced due dates. From an economist’s perspective, the rational thing to do would be to put all three due dates on the last day of class, because then you could still plan to do it evenly throughout the semester, but you would have no cost of doing that. So if something came up, you could postpone it. But what happened is the people who put all their due dates at the end of the semester ended up procrastinating, turning in work later, the papers were shorter, they were lower quality, and in general, they didn’t do quite as well. So that’s one study, I often will cite to students when we talk about due dates and deadlines, and so forth. But it’s an interesting study. And I haven’t seen anything else in economics journals, at least, related to that, but I’m sure there’s more that I haven’t seen in the literature.

Michelle: Fascinating stuff from across the disciplines.

John: One of the things you talk about in your blog post is that the strategies that students use for learning are not the strategies that evidence tells us are most effective. Students tend to use strategies that provide some short-term benefit, and seem to be easier, rather than the strategies that require them to struggle a little bit more with the content. One of the things you talk about are some ways that we could encourage students to adopt strategies that may not feel quite as good in the short run, but result in more learning. How can we motivate students to use evidence-based learning strategies?

Rebecca: …motivate them to struggle? [LAUGHTER]

Michelle: Right, but that is really what we’re talking about here. And I do want to go into this… big qualification here… I don’t think that students are just out there wanting to get the best grade and for the least effort necessarily, that’s just not a narrative about students that I buy into. So, I don’t think students are trying to do low-effort strategies. But, just like the rest of us, we don’t have a very good or accurate view always, a very empirical view, of what actually pays off in terms of learning. It’s pretty rare that we sit down and actually kind of do the math and say, “Well, I did this, I systematically changed the approach in this way, and here’s the outcome.” So we don’t come at it that way. So no wonder that over time, we end up with kind of a distorted view of what actually does work. So that’s a big piece of it. And it is true at the same time that these strategies we’re talking about… well, let’s take one, for example, of blocked study. Now this is a term that I also want to unpack a little bit too, it’s not super intuitive. So this has to do with the principle of interleaving, which I always say it doesn’t always apply in all studying, but to cases where you’re learning how to apply different problem solving strategies and you have to choose from several when you’re having to categorize and learn to discriminate among categories. So that is a subset of what students are sometimes learning. And the thing is, we have this great powerful line of research that shows that actually mixing it up in an unpredictable way, the different problem types or category types, means it’s going to be a lot more memorable when you actually work through those practice problems or practice sets. And if that’s the case, the unpredictability of like, well, something’s gonna pop up categorizing different painting styles, I have no idea [LAUGHTER] if it’s going to be a Renoir or a Monet, what could be next? It’s that unpredictability. So people sometimes confuse it with just like mixing up topics or having variety, but it really refers to the systematic principle. Now, when students are offered the opportunity to structure their own study, what do you know, they tend to go with blocked study, and again, it’s not because there’s some dispositional factor, they don’t want to do their homework or something like that. Really when you look at it intuitively, it’s like an illusion, block study feels so effective. I’m going to work through all of this one painter or all of this one way of solving a statistics problem. And then I’ll go into the next and our textbooks are organized that way, too. So students have seen that, and so that’s what they fall back on. And there’s some recent work that I’ve talked about in that blog post and in a few other places, that has really studied in a very granular fashion… it’s presented students with different alternatives, like here’s a blocked study schedule. Here’s an interleaved one. We don’t use the technical terms, we show them both options, and say, “Okay, let’s pretend you have a math test coming up. Which one of these do you want to do? And why?” And yeah, students, they gravitate towards the blocked one. And they say, I perceive that this is going to be easier, first of all, and that’s fine. We want to use the most efficient strategy. So they say, this is gonna look easy and also, it feels more effective. Because I feel like, “Oh, yeah, I’ve got it.” But as we all know, sometimes that’s a false sense of security. So that’s the example as it lays out in that one case, and I think that that is a larger kind of big dynamic, that we do have to be aware of what looks easy, what feels easy, what looks effective, what feels effective. Sometimes, your brain is kind of playing tricks on you. And that becomes very serious when it is the case that things like interleaved study are more effortful, but they’re going to pay off more for the time invested. Retrieval practice, which I and a lot of other folks in the space talk about so much, that’s another that it’s gonna require a different level of effort and engagement to close my book and say, “Alright, instead of rereading this chapter, what did I actually get out of it, or maybe I can seek out a quiz.” And to me, I also think it’s not just the effort involved, and research, by the way, it’s also showing students also to look at this and go, “that looks difficult.” It also kind of emotionally, I mean, I was feeling okay about this chapter, and now, I can’t really kid myself any more. So to the extent that students might be kind of saying, “Yeah, yeah, I’ve got it, I’ve got it.” …like, we might all do this, this will kind of bust that unjustified optimism, and that doesn’t always feel great in the short term. So if that’s the case, I mean, we can set up these wonderful learning activities, and if students aren’t going to do them, then they’re not doing them or us any good or any benefit. So, that’s the case as well. And so if they have a sense of the value, and a lot of these strategies that maybe we can touch on, do have to do with exposing and revealing and convincing about the value, and finding ways to draw students into exactly those techniques. So just because these are difficult, I do want to make sure everybody doesn’t get this terrible impression of like, “Ah, studying is going to be this miserable slog, no pain, no gain,” …it’s more subtle than that. And they really do work, you really are gonna get so much more out of the time that you put in and for students who really are stretched really thin as we’ve touched on. That’s an important powerful message.

Rebecca: You mentioned a number of specific examples in your post, do oyu want to dive into some of those and share something like the snowball effect or self-determination theory, or some of your other really awesome examples?

Michelle: Oh, thank you, I appreciate that. And, after all, the big philosophy and approach matters, but let’s get down to the actual techniques. So I’ve referenced something called the “snowball effect,” and this is just my informal term, but really, the more you know, the more that you want to know. And the more that you know, because of the way memory works, the more you know about a particular domain or subject area, the easier it is to acquire new facts. And like I always throw out the example of folks we know who are just really committed to some hobby or area of interest, the sports fanatics and so on, they can run into a fact one time and boom, they’ve got it, they’ve maybe got it for life, and they don’t have to study [LAUGHTER] or do anything like that. And so there is that snowball, or rich get richer effect, just because of some factors about how memory is set up and how it works. From a very practical angle, like I ran into this really intriguing study. And it’s not one where we’ve got piles of research yet, but this really got me thinking. So they did a study where they had students through retrieval practice, learn some basic facts about an area that they picked. And they were able to systematically track that when students did learn this foundational information more solidly, then when they had the option, “Oh, would you like to know more about this subject?” …students were more likely to say yes. And that’s totally voluntary. And that’s the sort of thing that makes our hearts go pitter pat, as teachers we want students leaving and going, “Oh, my gosh, now I really want to read that next reading that Dr. Kane assigned, for example.” And like when you were talking about, students are coming into class, and we’re trying to get them into the next level of conceptual stuff and exciting things they can do, if they don’t have the facts, it’s gonna be really, really tough, so it also really points up the importance of doing that. And it also, I think, addresses one of these big myths about memory. And this is one that I’ve talked about in some of my recent workshops, and so on, and I mentioned in my last book, this big myth that if we do focus on having students concentrate on remembering foundational information, we’re going to turn them off of learning: “Oh, it’s going to be this sort of these nightmare of drills.” And, “sure, they’ll know it for the test, but they’ll walk out and they’ll never want to be engaged with the subject again. So it’s a big loss, [LAUGHTER] right?” And this is really calling that into question, saying that sometimes knowing some of these cool initial facts can start to set you down that path and then maybe someday, you will be that expert who can hear a fact one time in this area and we’ve got it. Why? Because we already know so much about it. So again, that whole Interplay there. There’s also the role of choice and autonomy. And this is one that I think a lot of really intuitive, committed, teachers really hit on early, even if they never really have some of the more formal terminology for it. So when there’s choice, not only are students more invested in what they’re doing, there’s possibly a role of curiosity here. So I talk about, in this blog post, this sham lottery study, it was one of these, where if you look at it on the face of it, you’re like, “What are they doing?” But as a psychologist, I’m like, “Ah, that’s really clever.” So basically, they had research participants going through this little pretend lottery of like, “Okay, you’re selecting out of this bucket of red balls, and so on. And what do you think it’s going to come up?” And the one twist that they put in there, is sometimes people chose which of these two little buckets, there are these little random drawings, that they were going to focus on? And then it’s like, okay, well, we can either just move on with the study, or you can see how it came out? Well, they want to know how it came out when they chose… even this incredibly arbitrary [LAUGHTER] low-stakes situation. So I think that’s also another kind of natural, emotional process, motivational process, that we can tie into… setting up curiosities or questions, but also having students say, “Well, which of these two projects do you want to do?” These days I offer options whenever I can. Would you like to write a term paper? The sort of formal paper? Or would you like to put together a slideshow that you can narrate and share? Big learning objectives are probably similar, but students can pick and I always present in practical terms, I say, “Well, if you are going to graduate school next year, and you need a writing sample for your portfolio, this is a great opportunity to do that orr if you’d like to stretch your skills with oral presentations, maybe because of the last few years, you haven’t gotten to do that as much, then you can choose this,” …but simply by having them make a choice, the research would predict that they are going to be more invested, and they’re going to be involved in these more effective things. So that’s one and oh, I’m really excited to see what’s coming out in this whole sub area of “Okay, we’ve done all the research we know that things like retrieval practice and interleaved study, all this engaged stuff. We know it works. We put it on a tip sheet, we gave it to students, nothing happened.” Uh oh.[LAUGHTER] Now what? So not just the like, “Okay, what should students be doing?” But “how do we get them to do those very things?” And, boy, if there was ever a time when we realize, yeah, my ability to just sort of exhort you and make you do things because I say so is limited, this is the time. Because I don’t get to go home with students [LAUGHTER] and say like, “Alright, that whole thing about quiz yourself. And so now you really have to do it.” So there’s this relatively new framework that’s come out too from some cognitive psychologists that I really admire, Mark McDaniels and his team. The knowledge, belief, commitment, and planning approach, KBCP. So this pulls in from some other research on intentional behavior change that’s also been perking along just all on its own for years and years. We know so much about how people set a course and decide to change their behaviors. And study skills are, after all, kind of an entrenched pattern of behavior for many students by the time they get to us. How do we go in and change it? And yeah, it’s absolutely not through my least favorite technique, which is put together a list of random tips and hope for the best. So they say, alright, knowledge is the first step. So just telling students like, “Hey, there’s all this research that shows that if you close the book and quiz yourself, you’re going to get more out of this. If you do the reading quiz that I set up for you, and do it as many times as you possibly can, that’s going to help you retain the foundational information.” I’ve told them, I’ve shared it with them, or something like interleaved study, if you’ve got different problem types, mix them up. But it doesn’t stop there. That’s only the first step. So the next step is belief. And that means changing beliefs, which means persuasion. So we kind of dust off a whole bunch of other things out of the psychologist’s toolkit. How do you persuade people? Well, you show, don’t tell. So this team proposes doing things like “Well, let’s run a head-to-head comparison, like a Pepsi Challenge, in class. Sure, your brain tells you that you learned a ton just from reviewing, but did you? Let’s try it.” And this takes some time. I mean, this is not easy. But this is one of the things they propose: commitment. So now that I’ve persuaded you that this is the way to study, now, what’s your next step? So getting your students to say, “Yeah, I actually authentically believe this. And I see how it’s going to help me and I’m going to try it.” And then of course planning. So instead of just like, “I will do this,” right? Those of us who are veterans of New Year’s resolutions of yor [LAUGHTER] know that that is not the way to go. So yeah, saying “Okay, but here’s what I’m going to actually do. So I’ve got a test coming up, I’m going to maybe set up a study schedule, instead of just cramming it all in the last minute, which is [LAUGHTER] a really good empirically grounded strategy. I’m going to find these practice quizzes, or maybe I’ll get together with a study group and do that. So here’s my plan.” And then if possible, circling back and say, “Well, did it work.?” And hey, if we’re right, then students will actually try it, they’ll say, “Wow, in less time, I knocked the top out of this test that I was really worried about.” And that is going to feed that virtuous cycle of going right back to those effective strategies. So KPCB, I love it too, because I’ve been doing something similar in the Attention Matters Project that I think I’ve talked about on some previous episodes as well, which is all about having students themselves come in and see how their attention is limited, learn about the effects of things like distraction on their learning. But we don’t stop there. We give them a few rudimentary tools as well, we say, “Okay, what is going to be your plan if things are dragging in class and your mom is texting you? That’s tough. How are you going to get through [LAUGHTER] that without then checking out of your class? What are you going to do if your neighbors are watching who knows what on their laptop or they’re texting and it’s bothering you? What is going to be your plan?” So getting students to really think ahead to those things, commit to doing them in a way that works for them, and puts that newfound knowledge into practice. So those are some of the things that I’m really experimenting with and excited about right now.

Rebecca: So in that approach, it seems really necessary to help set up a structure for students and then circle back and have a reflective piece so that maybe they will do that on their own next time.

Michelle: And there’s some exciting suggestions from research here, too. I mean, I know it’s easy sometimes as faculty, especially at the end of a long year, like this one, to say, “Ah, well, did it actually stick with them?” But there’s a couple of different projects out there that have kind of converging on this idea that once students really do see something like retrieval practice, active studying, and so on, and once they really experienced that, as part of the structure of one course, they absolutely will run with it. So they will go into the next class, whether in your discipline or not, and say, “Well, from now on, I’m actually going to have a study plan that’s set up in this particular way and I’m going to do this.” So I personally find that very, very encouraging that “Yeah, it takes some work to do this stuff, but the payoff, even if you personally don’t see it right in front of your eyes, the payoff is likely there.”

John: And so the more faculty you start doing these things too, the more likely it is that students will adopt new approaches. So spreading this more widely is helpful.

Michelle: Yes, yes, a hearty I agree to that statement. And I can test on my own campus, I’ve seen more faculty bringing in more structure, things like online reading quizzes, I have noticed that, so I guess that’s a counterpoint to the “Wow, my lived experience is telling me that there’s these issues in engagement. Maybe so, but my lived experience is also telling me that students are coming to me more ready to be proactive about their study, they need a little less persuasion to do things like reading quizzes, because they at least they’ve seen them before. So yeah, I think it absolutely can work that way you’re describing.

Rebecca: So we always wrap up by asking, what’s next?

Michelle: Wow, well, these days, I’m working a lot on my substack newsletter, it’s called the R3 Newsletter. And I think this is the mechanism by which we connected on some of these new topics. So I love that it’s already starting these great dialogues. And if you haven’t seen substack at all, it’s a bit of a blogging platform. And my substack is free, some subtasks are paid, but mine is definitely free. And so, for example, if you’re interested in this topic, I would definitely tell your listeners to check out Sarah Rose Cavanagh’s substack, as well, it’s called “Once More, with Feeling” and I also want to affirm that she is and her work are really at the forefront of this whole topic of motivation and emotion and particular in learning so a great other substack to follow it and buy her book, Spark of Learning is also just an absolute modern classic in this. So I decided to get in the fray since I saw these wonderful thinkers around me also doing this. And this has been a really good platform, and a way to structure for myself something that I felt needed a refresh, which was my reading of the literature that’s coming out. So what I do is twice a month, approximately, I’ve been putting out discussions of research that I’m reading. My little heuristic is anything that was published in approximately the last year, really privileging the new stuff. And I’ve historically just seen that then when I really get into the nitty gritty of the research and what it says and what it doesn’t and parsing that for folks, especially those who may be outside of social sciences, that’s where I get the most affirmation from folks and people saying, “Yeah, this was really helpful.” So I decided to run with that. And so that’s a big project ight now. I’ve been really happy with the reception and just working on that. I’ve been writing and thinking more about this topic of motivation and cognition. So as we mentioned at the top of our conversation, it’s one that goes back to kind of my initial ponderings, thrashing around as a beginning graduate student of like, “How does this all work?” Coming back to that, and really finding new ways that I can share that with my fellow faculty. So getting the word out there. I have a few new projects that I’m working on that tie back to that attention issue. So that’s another perpetual area of interest for me. So I have a few new writing and research projects that are going on with that and kind of in the development phase. And this summer, I am going to be catching up on a stack of books, just an epic number of these great books and works that are coming out. Seems like every week, there’s a new thing that goes on that list. So I look forward to a few weeks or more to really concentrate on that.

John: And we should note this is the second podcast that has come out of things we’ve seen posted on your substack blog.

Michelle: Oh, wonderful.

John: One other thing. Sarah Rose Cavanaugh has a new book coming out that we were fortunate enough to get a draft copy of and it should be out this summer: Mind over Monsters, if you’d like to see more about this topic.

Michelle: Oh, absolutely. Alright. It’s on the stack now.

Rebecca: That pile keeps growing.

Michelle: Yes, it does.

Rebecca: Better add on another week. [LAUGHTER]

John: We could all use an extra week or summer.

Rebecca: Right? Yeah.

John: Well, thank you. It’s always great talking to you, Michelle.

Michelle: Oh, my pleasure, you as well.

Rebecca: Thanks, Michelle.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

222. Interleaved Practice

Students engaging in blocked practice focus their efforts on a particular topic and then move on to the next topic in sequence, resulting in a perception of content mastery. Interleaved practice provides an alternative approach in which students engage in learning activities that require them to determine which concepts are relevant in a given application. In this episode, Josh Samani and Steven Pan join us to discuss their study comparing the effects of blocked and interleaved practice on student learning.

Josh is an Assistant Teaching Professor of Physics at the University of California, Los Angeles. He is also an instructional consultant for the Center of Education Innovation and Learning in the Sciences and Director of the UCLA-APS Physics Bridge Program. Steven is an Assistant Professor of Psychology at National University of Singapore whose research focuses on evidence-based teaching approaches.

Shownotes

Transcript

John: Students engaging in blocked practice focus their efforts on a particular topic and then move on to the next topic in sequence, resulting in a perception of content mastery. Interleaved practice provides an alternative approach in which students engage in learning activities that require them to determine which concepts are relevant in a given application. In this episode, we discuss a study comparing the effects of blocked and interleaved practice on student learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Our guests today are Josh Samani and Steven Pan. Josh is an Assistant Teaching Professor of Physics at the University of California, Los Angeles. He is also an instructional consultant for the Center of Education Innovation and Learning in the Sciences and Director of the UCLA-APS Physics Bridge Program. Steven is an Assistant Professor of Psychology at National University of Singapore whose research focuses on evidence-based teaching approaches. Welcome, Josh and Steven.

Josh: Thank you.

Steven: Thank you.

Rebecca: Today’s teas are… Josh, are you drinking tea?

Josh: I’m drinking the purest kind of tea which is no tea at all. Water.

Rebecca: It’s important to tea to have water, so this is good. This is good. How about you, Steven?

Steven: I’m having some very nice chamomile tea in honor of the podcast.

Rebecca: Awesome, even though it’s midnight.

Steven: Yeah. well, non caffeinated

John: …and I’m drinking Irish Breakfast tea today.

Rebecca: I have a classic English breakfast today as well.

John: We’ve invited you here today to discuss your research project on interleaved practice in physics classes that was published in Nature Science of Learning in November 2021. Before we discuss that study, though, could you describe to our listeners what interleaved practice is and how it differs from blocked practice?

Steven: Interleaved practice is a way of arranging learning activities in which you switch between different topics or skills as you try to learn them. So for example, say in the domain of mathematics, you’re learning to solve volume problems, graphing problems, factoring problems. With interleaving, in a single session, you would switch back and forth between those different skills. So maybe you will attempt a volume problem, then switch to a graphing problem, then a factoring problem. You’re constantly switching back and forth, mixing it up, or interleaving, between those different skills or topics. Now in contrast, a more common practice is known as blocking or blocked practice. And that’s where you focus on one skill or topic at a time, sort of exclusively. So for instance, in an initial session, you’ll focus only on, say, volume problems. And then it’s not until the next session, you will only focus on graphing problems. And then the next session after that maybe only factoring problems. So blocking and interleaving are in a way sort of diametrically opposed. In one case, you’re constantly switching back and forth. In the other case, you’re focusing on one at a time. And currently blocking is much more common in everyday practice in classrooms, it seems to be better organized, more intuitive to people, whereas blocking, firstly, most people don’t know about it. But if they hear about it, they may think “Oh, that seems disorganized, haphazard. Why would I even want to do that?” And so interleaving and blocking… very different ways to schedule your learning.

Josh: And I think also, from the point of view of gaining mastery, there’s this intuition that if you want to master something, you kind of need to block your practice on each sub skill, and hammer away at that for a long time until you master that sub skill then move on. We think this is maybe another reason why, in general, people intuitively are drawn to blocking as an initial study strategy.

Steven: Right. And just this idea of practice makes perfect, it kind of sometimes seems to imply that you should focus and target a particular skill or topic and just become really good at it by practicing it over and over. And that’s the essence of blocking.

Rebecca: Can you talk a little bit about some of the prior lab experiments, and what they suggest about interleaved versus blocked practice before we jump into your current study.

Steven: So, the first studies of interleaving were conducted in the domain of motor skills, and this is work dating back to the 1980s. And the chief finding is that… and it was surprising at the time, and still seems somewhat counterintuitive today… that if you interleave as you’re practicing a set of motor skills in a given domain, that results in better overall proficiency than blocking. For instance, in the domain of badminton, learning to serve different types of badminton serves, the finding has been that if you interleave as you practice, and you practice one type of serve, then you switch to another… then another… then another… that results in better serving ability than practicing only one kind of serve at a time before switching to the next. And that finding that interleaving benefits motor skills has been shown in a variety of motor skill domains, including baseball, basketball, and so on. So the earliest work was in motor skills. Moving on to the early 2000s, there was a move into non-motor skill tasks. And this is primarily involving perceptual category learning, where you’re shown, essentially, pictures that exemplify different categories, for instance, bird families, different artists, painting styles, and the finding there has been that if you interleave as you’re viewing these examples, you view one example from a given category, then you view another example from yet another category, you keep switching… that results in better ability to recognize those categories than if you block where you’re just seeing one category over and over at a time, and that finding has been especially the case for categories that are sort of confusable, similar to one another. And it seems that interleaving is a great way to learn to tell them apart. And now, even more recently, there’s been a move to really highly educationally relevant tasks. For instance, I myself have published a couple of studies investigating interleaving with learning grammar rules in a foreign language. And it seems that interleaving is a great way to learn different sets of grammatical rules. Perhaps the most sort of impressive set of recent studies has been worked by some researchers at University of South Florida, in which they interleave math homework in seventh grade classrooms. So early, I mentioned learning to calculate volumes and graphing and so on. That’s literally the kinds of problems that they interleaved in homework in seventh grade classrooms and that results in greatly improved learning over the traditional blocked approach. And so, overall, the literature on interleaving is small but growing, but there have been a variety of promising results across very different domains. But, really, the literature has just scratched the surface and it’s not yet clear whether the benefits of interleaving are universally applicable. But again, it’s showing considerable promise.

John: Your specific study took this out of the laboratory and into the classroom. Could you describe the experiment that you did?

Josh: Yeah. So in the winter quarter of 2020 at UCLA, I was teaching two sections of Physics 5C, which is a large-enrollment Physics for Life Sciences course. This is the course that pre-meds take mainly, but also some other students from the life sciences. And each course had an enrollment of about 180 students. So these were pretty large classes, so we can get some good statistics for our study. And the topics are things like electricity, magnetism, quantum mechanics, all of these things are taught at somewhat of an introductory level. These are not like physics major, really in-depth courses, but nonetheless quite challenging. And most students in this series find these topics quite challenging. And what we did was, we gave the students three homework assignments per week that were spaced out basically as evenly as we could, given the course schedule, and in one section of the course… so there were these two 180-person sections. In one section of the course,for the first month, all of their homework practice on all of these assignments were blocked practice. Whereas in the other section of the course, all the homework assignments were interleaved practice. So what that meant in our study design is that on a given homework assignment, students in the blocked practice condition, for every given topic, they would receive three problems that were what we call isomorphic problems. So isomorphic problems are problems where the surface features are quite different, but the deeper underlying structure is quite similar. In the blocked group, students saw these isomorphic problems all blocked together. Whereas in the interleaved group, students saw these different isomorphic problems for a given topic arranged so that they spanned multiple homework sessions. So instead of seeing all those problems in a single homework session all in succession, they saw them in three different homework sessions that were distributed across a week. At the end of this month, all students were given a surprise test to see if their learning had been retained and if they could transfer their learning to somewhat more challenging problems, in some cases for some of the problems on the test, and then basically, we measure the difference in performance between the blocked group and the interleaved group on this assessment. And then for the second month of the class, we did exactly the same thing with the new topics, but the section that was previously interleaved was now blocked, and vice versa. The idea being that, this way, every student in these two sections received both of the conditions. And we could make sure that regardless of which students are getting what type of problem set, we see the same result. So basically, like an internal replication almost. So that’s basically the design of the study.

Rebecca: So the intervention was the homework. Were the things that you were doing during class time blocked practice, or were those also interleaved practice?

Josh: Yeah, that’s a great question. So actually, maybe I should also clarify that the only thing that was different about these courses was the homework. Everything else was identical to the extent possible. I was teaching the course, so the lectures were back to back and I attempted to make the lectures as identical as possible. And lectures were somewhat traditional in the sense that there was a lot of me talking about generalized concept and abstract principles, but also doing some worked examples, and a bit of audience interaction. Students also had lab sections that they attended each week. They had discussion sections where they worked on problems. But all of those things are exactly the same. The only thing that was different, literally, was the arrangement of the problems. And in particular, the entire group of problems, the blocked group and interleaved got were exactly the same. So it was literally just the ordering of problems and their distribution and nothing else.

John: So why did you use surprise tests rather than embedded assessment?

Josh: Yeah. So in an authentic educational context like this, where you’re trying to measure the effect of some intervention, it’s pretty well known to anyone who’s taught a class in any context whatsoever that students tend to cram they’re studying. And the problem with that is that you could imagine,we do this intervention in the homeworks and we change the ordering, and students do all this stuff that’s different. And all of a sudden, if they know a test is coming, they can study 10 or 20 hours. And we have no idea what they’re doing during that studying, maybe everyone will interleave their practice during that studying. And it’s really quite a large amount of studying. So maybe it’ll just wash out any effects that we would have seen otherwise. So we use surprise tests, basically, to eliminate the contamination of cramming, which we know is ubiquitous. This way, we can measure, with much more precision, the effect of the thing that we did, as opposed to it being contaminated by cramming. And, by the way, it’s also pretty well known in the science of learning literature that cramming does actually work. So when students cram for 20 hours, we expect them all to do pretty well, as opposed to when they don’t, we can kind of see what the homework actually did for them.

Steven: …with the caveat that cramming is effective, but short lived in terms of its benefits. And just to add on to what Josh said, we thought of the surprise test as sort of the purest measure of the effects of our intervention. It was uncontaminated by the cramming and other various things that students may have done in the run up to the announced high-stakes exams.

John: That was something I’ve really liked about this, because it’s a methodology that I think perhaps could be used a bit more in measurements of teaching interventions. I haven’t seen it used very often in the past.

Steven: It’s not as frequently used as perhaps it should be, but there are some studies in which surprise exams are implemented just for the very reason that we did, sort of to have an uncontaminated assessment of whatever was being done in that particular study. One thing I also would add, that I think Josh mentioned… that students were genuinely shocked when they came into the lecture hall and rather than a review session, as we had suggested they were going to experience, he started handing out the surprise exam forms

John: It’s a form of retrieval practice, which is a form of review.

Josh: Oh, yeah, I think, in fact, I’ve actually used that as a form of review in the past, interestingly enough, even before we did this. Another thing I want to add is that using high-stakes, for example, midterm exams to measure the effectiveness of these sorts of things, is also problematic in the sense that, presumably in education, what we’re trying to measure is long-term retention, to a large extent, and students cram on these tests. And so given what Steven said, since cramming is effective only in the short term, what we’re really measuring is just what students know in a very short period of time. And I think it’s more broadly important to move away from that protocol. We should be measuring things at larger time intervals in general.

John: So, what did you find? How large were the effects?

STEVEN So in both stages, first of all, students who had received interleaved practice did significantly better. So, both from the perspective of it being a statistically significant finding, but also in the size of the effect. In stage one, the effect size was 0.4, which for those who maybe haven’t used such measures before, that would be considered like a medium-effect size, a moderate size of an effect. But in an authentic educational context, like in the wild where we’re doing this experiment, it actually might be considered a kind of a large one, because in general effect sizes, not in the lab, for studies that are done in a setting like a classroom, effects tend to be somewhat smaller, because it’s like all the different things that are going on. But in stage two, the effect size was 0.9, which means this is a very large effect, especially by the standard of educational interventions in classrooms. And just for those who, like I said before, effect size is not something their familiar with, in stage one, tthe average performance for blocked students in stage one was 0.43, which means that they answered 43% of the rubric items correctly, whereas in the interleaved class, it was 0.54, so they answered 54% correctly. And in stage two, the blocked students answered 27% of the rubric items correctly and 47% in the interleaved group. So it was a larger difference.

Rebecca: I’m curious about the second stage in that, if they’ve already been surprised one time, were they then primed to get a surprise a second time and maybe motivated more effort and time on homeworks?

Josh: So interestingly, one thing we did in the study, which is part of the published data, is we asked students to report how much study time they devoted before the surprise test. We also asked them to report how surprised they were that there was a surprise test during the review session. And interestingly enough, the surprisal remained very high the second time, and the amount of studying students did before the surprise test beyond the homework was still quite low. So at least from the self-reported measure, we can be pretty confident that things were pretty similar in both of the stages in that respect.

Steven: Perhaps though, if we did this a third time, maybe that would no longer be the case, but it worked both times.

John: Some of our listeners might be wondering how you counted the surprise test? How did you include this in the computation of their grade.

Josh: So in the end, actually, this had no effect on the computation of their grade. When the surprise test was administered, I did offer 1% extra credit to students based on their performance, but it didn’t actually affect the core grading rubric. So we wanted to make sure that if there were, in fact, big differences in performance, it wouldn’t adversely affect anyone in the long term in terms of their grade.

Rebecca: After having this intervention twice in the classes, sid you talk with students about this practice and its impact on them?

Josh: You mean students in those particular classes?

Rebecca: Yeah, after the fact.

Josh: I actually have not. That’s an interesting question. Actually, I haven’t even really seen very many of them to be honest. I teach so many students that it’s very rare for me to interact with anyone from previous classes, because everyone’s so busy. There were one or two instances of this, and they were very surprised and interested, actually, to see the results, but nothing particularly large scale.

Steven: One thing we did do, though, was for the homework assignments, have students rate after they completed those assignments, how well they thought they had learned and so on. And in those ratings, they didn’t seem to show any evidence that one particular homework type, namely interleaving, would actually be better than the other type.

John: Why might interleaved practice have this sort of positive effect on longer-term recall.

STEVEN So there are a variety of viable sort of explanations for the benefits we observed here. And I like to sort of categorize them… actually, Josh and I like to categorize them… sort of into two main categories, first, being interleaving, sort of engaging long-term memory processes, and that enhancing learning, and then the second being interleaving actually promoting specific kinds of learning that are unique, that you don’t get when you engage in blocking. So, with the first, it’s based on the idea that, with interleaving, when you encounter a given problem type, you don’t revisit it again until some time has passed, there has been some spacing or distributed practice. And in the learning sciences literature, distributed practice (or revisiting information after a delay) is actually a very potent enhancer of long-term memory. So again, with interleaving, you see one problem type, and you’re practicing other problem types, you don’t return to that original problem type until sometime later. And so, because of that spacing, the next time you encounter something, the students are going to have to recall from long-term memory, what they remember about that particular problem type in order to solve it, and so that long-term memory that’s constantly being engaged by interleaving, may be enhancing learning. So that’s sort of the first explanation. And the second set of explanations really involves interleaving because you’re switching back and forth between these problem types. This may afford you the opportunity to sort of compare and contrast one problem type with the next. And that can yield a host of possible benefits. You may become better, for example, at going beyond surface features, but actually recognizing what really distinguishes one problem type from the next. So on the surprise exam, you actually can tell this problem type from the next. You can also perhaps get better practice at choosing the right strategy to solve a given problem versus another. And also, our surprise tests had some problems in which students had to combine strategies from multiple problem types in order to come up with a solution. And so it’s possible that with interleaving, because students were switching between problem types, that might have given them the opportunity to think about what relates one problem type to the next, and that could have enabled them to be better able to combine strategies to solve these surprise test problems. None of those kinds of learning are possible with blocking, because you’re focused on one problem type at a time, there’s no switching, no ability to compare and contrast, and so none of those learning opportunities occur. So those are all possible explanations for how interleaving is enhancing learning in this particular study. And it’s also possible that multiple mechanisms are at play here, resulting in this powerful benefit.

John: In some of the lab experiments involving interleaved practice, generally, when students were exposed to interleaved practice, they felt they were not learning as much. When they were exposed to blocked practice, they thought they were learning more, yet the longer-term testing results in most of those experiments suggests that the opposite was true. Why is it that we so often see this sort of metacognitive illusion, where students perceive things that are beneficial as being less helpful, while the types of strategies they most typically use as being more useful?

Josh:: Maybe I can first of all say one thing just about what we actually measured in terms of judgments of learning very specifically, I think Steven did touch on this a bit earlier, but it might be interesting for the listeners to hear that we’ve actually measured two things. So we measured perceived difficulty of the homework. So right after students worked on an assignment, they rated the perceived difficulty, and we also measured judgments of how much they learned during that homework assignment, on that particular set of concepts. And in those two cases, interestingly, students tend to view blocking as being easier, which maybe actually is pretty intuitive because when you kind of are doing the same thing over and over again, you get this fluidity with it and you get into the state where you kind of know what you’re doing and you don’t have to think too hard about it. And I think that probably is what contributes to some degree to what you’re saying… why students don’t maybe respond so well to interleaving. Because you don’t get that fluid feeling of, “Okay, I’m kind of rolling, I’m doing these problems, and I know how to do them all.” So they perceive them as being easier. And they also felt like they learned more… possibly also, because they’re conflating this sense that they’re really fluid with how much they’re actually learning, which it turns out may not actually be the case, and probably isn’t the case. And in fact, that’s what we found here, is that. in both stages, this is what we found in terms of the perceptions. But in both stages, students did better on the surprise test. But also, interestingly enough, students in the blocked group did do better on their homework in general than students in the interleaved group. So it was easier for them. So in some sense, they were accurate about how hard the homework was for them. But they were inaccurate about what that meant in terms of their learning. And maybe I’ll let Steven go beyond what I said about why this may happen. But that’s one idea.

Steven: Yeah, I agree with that interpretation, I would add that one of the sources of metacognitive illusions relating to effective learning strategies involves students, learners, sort of conflating performance with learning. So how well you were actually doing in the moment with how much you’re actually learning sort of underneath everything. And, with blocking, and a variety of learning techniques that are actually less effective of the long term, they can give you high levels of performance as you’re using them, sometimes because they make the learning task much easier. And in our case, with blocking, because your practicing the same type of problem over and over, perhaps after the first problem, you no longer really have to work all that hard, you just repeat the same procedure… performance level is high. So students may infer that oh, well, I’m doing really well, I must be learning quite effectively. But actually, the underlying learning is quite low, whereas with interleaving, the performance level is not as high because they’re switching, it’s more difficult, they may then infer that, “Oh, my performance is low, I’m not doing so well, I’m not necessarily learning as well.” But actually, that struggle is very beneficial for learning over the long term. But this sort of inability to differentiate between learning versus performance, because performance is not always a direct indicator of the underlying learning, may be the source of this sort of disparity.

Rebecca: I love that you’re bringing this up, Steven, because I think a lot of times students or even folks that are new and in a professional position too, that are just starting out as an emerging whatever, whatever field they’re in, feel like they need to perform. And there’s this pressure or grades are often assigned based on performance, rather than the idea that learning and growing requires some struggle. And we don’t always think about that or encourage that.

STEVEN Oh, absolutely, yes.

Josh: Yeah, I think this has some implications for student mindset in general. We talk very often these days about growth mindset, and students having growth mindset being beneficial. I think students internalizing these sorts of results, although it’s very hard to get students to do that, may go some way in convincing them or helping them generate a mindset for themselves of growth, as opposed to a fixed mindset. So, understanding that things that actually are effective for learning oftentimes feel very difficult and lead to lower performance.

Steven: And I would just add on that, some of the more difficult but more effective in the long term learning techniques, they result in more mistakes and more errors. And certainly with interleaving, we saw a greater rate of problems being solved incorrectly than in the blocked condition, so students were making more mistakes. But the act of making those mistakes was not necessarily detrimental for learning in the long term.

Rebecca: Now, we can just get our assessments and things like this to match with that idea…

Josh: Yeah,

Rebecca: …we’d be all set. [LAUGHTER]

Steven: That would solve all our problems.

John: A lot of it, I think, is just the difficulty of convincing students of these issues, that there is benefit from making mistakes, and there’s some substantial learning gains from that. And it’s something I’ve been trying to do in my classes for a long time, without as much success as I’d like. Students like to feel good about their learning, and they like to be successful immediately. But that’s not consistent with long-term recall, or transfer ability.

Josh: Before I forget, along these lines, in terms of student perception, and judgments of learning, there’s kind of a couple of funny anecdotes along the following lines. So, when I was giving the surprise tests, students are shocked. And you can kind of hear this audible thing in the room with like, “Oh my god, we’re doing the surprise test…” and increasing anxiety and all these things. And something that I noticed, and you have to be, of course, very careful about these sorts of anecdotes, but I think it’s possibly indicative of something deeper. What I noticed was, in the blocked conditions, when they were taking the surprise test, there actually were a number of students who immediately decided to give up and turn in their tests. And basically unsolicited said to me, “I haven’t done any studying, how could I possibly answer any of these questions?” And so there’s this interesting thing where students somehow perceive the practice that they were doing as being completely useless. There were a few students in the blocked condition, in particular, who felt this way. And they only felt that after having crammed for an exam, would they be prepared. So there’s this kind of interesting, just also cultural, sort of perception around practice where somehow this spaced practice that they were doing this entire time, was not really doing anything for them and they had to cram to do well on the test. So I think it just speaks again to these kind of conceptions student have about learning, and oftentimes these illusions about learning that are really detrimental.

Rebecca: Yeah, I was just reflecting on what you were saying, Josh, and that I’ve definitely had the same experience with students. I teach web design. So I teach something very different. But we might work on a new project or a new assignment, and so we’re revisiting stuff from the past over and over again, because to start a new project, you have to know the things that we did before. And students will say, “Oh, I can’t do this. We haven’t done this in a while.” So just remembering many of these same instances [LAUGHTER] reflect the same idea. A lot of our textbooks and even adaptive learning platforms are primarily set up to encourage blocked practice, and many of our courses too, and what are some ways that faculty can build interleaved practice into our course design? …maybe beyond there’s the homeworks, as you mentioned, but are there other ways that we can start thinking about it in our practice?

Josh: Yeah, so firstly, one thing that any instructor can do that might promote kind of like an organic form of interleaved practice is to make sure their course is cumulative to a large extent. So if you tell students from the outset that you’re going to have a one midterm on so and so material that’s right before the midterm and this other midterm on so and so material, they’re basically going to focus on that material. And they’re not necessarily going to mix what they learned in that section with what they learned in sections for other tests. So if your course and your exams are cumulative, meaning that you are, for example, testing on a variety of topics, as opposed to just a single topic at a time, it’s a natural incentive for students to interleave their practice, because they need to do that for the exam. And in those cases, it’s probably a good idea to encourage students and say, “Well, you know, the structure of the test is going to be that it’s going to have mixed problems. And so it makes sense for you to mix your practice…” and you don’t even necessarily need to explain what interleaving is for someone to kind of feel like, it’s probably a good idea for me to mix my practice if the test is going to be mixed. Also, I know you said beyond the homework, but it actually can be logistically challenging to do interleaving If you haven’t thought about the sort of schedule you would need to use. I think the schedule we have in our paper actually is a good model for this because we had to think for some time about how you could actually do this effectively in a real classroom. And the main constraint to think about is that, with interleaving, if you’re mixing topics, you can only mix topics together you’ve already had before in the course, like something that students have learned something about. And so you just need to think sort of intentionally about: well, given the schedule of my class, and given the topics that we’ve covered, how spread out can I make the topics over the time that the students have to do the practice. So I’d recommend if people are looking for a concrete version of this, to see how you could arrange the schedule to make this happen. We have a diagram in our paper where you can see how they’ve been shuffled together, and it might be kind of helpful.

Steven: Yeah. And to add on to what Josh said, a lot of the work with interleaving is in the setup, sort of designing the interleave schedule. And probably one of the reasons why blocking is so popular is it’s frankly, much easier, you just: this week, I’m going to focus on this topic and once it’s done, I’m just going to move on. You don’t have to think about it again. With interleaving, the instructor beforehand, probably should sit down and think about: Okay, how am I going to make things cumulative? And then in the service of that, how am I going to perhaps arrange various learning activities where I’m going to require students to revisit other material? I’m going to perhaps mix things, randomize things in a way that is less predictable, less nicely, discreetly organized as a blocked schedule would be. And doing that would be a way to set up interleaving in advance and then just follow that schedule. I think it’s probably very difficult, possibly impossible, literally, to have interleaving sort of introduced midway through. You should plan it in advance and then execute it, and then all the difficult work is done up front.

Josh: And you mentioned also adaptive learning technologies. And at UCLA, for example, we use online homework systems that do automatic grading of assignments and problem banks and things like that. One really easy thing you could do if you use such a tool is just to try to shuffle the problems together from different topics. So we use something called Mastering Physics and on that platform, you can pick and choose the problems very easily, and you don’t want to think too hard about it. You could just make sure that you have problems from different sections mixed in as opposed to when you give a given homework only problems from this one section in the book.

Steven: Also explaining to students from the outset, sending out the expectation that interleaving of the materials will happen and it’s by design, because you could even say potentially that the exam isn’t necessarily going to be nicely organized, where you’re going to be told this problem is on Chapter one, this next problem is gonna be on Chapter Two, it’s probably going to be mixed together, so why not practice in a way that is similar to the actual exam. And moreover, in real life, everyday circumstance, usually, the next situation you encounter is not necessarily predictable and a repeat of what you just saw. Things are often interleaved in the real world. So in a way, engaging in interleaving is kind of a better proxy for what you actually have to do when you apply skills.

Josh: But I would say one last thing, just in terms of implementation, as with many of these sorts of things, you have to know ahead of time that there might be some student resistance. Because the homework is going to feel a little bit more difficult, you have to be prepared that some students are going to say, “Well, this is totally different than what I’m used to” …and maybe feel like it’s just the wrong way to do things. And so you just have to be prepared as an instructor, as an educator. We know this as educators, that we encounter resistance, but it’s a good thing to know ahead of time.

Rebecca: I think one of the key things when we’re introducing techniques that feel like a struggle, where we might fail is that there’s room to fail and not have long-term consequences in terms of grades and things and to think about making sure that we’re setting these things up in a way that promotes learning and promotes engagement with the method rather than just resisting it or just being really upset about how it might negatively impact a grade. Because also when we have emotions running high that are negative, like that’s not going to help our learning either. [LAUGHTER]

Steven: Right and making, I think, some of these activities low stakes, and not being highly detrimental to one’s grade if one bombs an assignments that’s interleaved, probably would be very palatable to students, but would lower that anxiety,

John: One of the things I’ve been doing for about a decade or so now is using progressively cumulative exams in my large class where I have weekly quizzes that a growing share of them are drawn randomly from earlier portions of the course and student resistance to that is pretty high, I do talk to them about the benefits of interleaved practice. But when they’re trying to work through the exam, and they’re trying to remember things from a month or so before, they don’t find that quite as pleasant. But reminding them that the more effort they put into this, the more they’re going to remember in the future, and that the goal of the class isn’t to get them the highest grades on their exams, it’s to give them a preparation in the discipline that’s going to let them be more successful when they take upper-level classes that build on this foundation. That’s what I tell them, and some students do accept that. But I wouldn’t say it’s a universal acceptance of this.

Rebecca: It’s so hard, John.

Josh: Yeah, who would have thought that the purpose of education is for you to remember things a month in the future?

Steven: It’s always a conundrum that perhaps the most effective instructors are not necessarily the most popular instructors. [LAUGHTER]

John: What does the evidence suggest about interleaving, across disciplines and across different types of concepts? Does it seem to work as well in all disciplines?

Steven: There seems to be a promotion of interleaved practice, particularly, I think, in the popular media that latched on to this technique and think, oh, wow, it’s really amazing and counterintuitive and cool. I’m going to recommend it broadly. And I think one of the concerns with that is then you have suggestions to interleave just anything under the sun and in any way. And although the effect of interleaving in controlled studies, in many cases, has been very, very promising, that isn’t necessarily a license to just interleave anything. And in particular, one of the sort of potential moderating factors is maybe the degree to which the materials that you’re interleaving are confusable, or similar with one another. And that may be really critical. So you can’t just interleave my say physics homework with history homework, and things like that. But the problem is then, when some people read these articles online, that say interleave, they don’t get that nuance. I think that’s a bit of a concern. And further, beyond that, there does seem to be different benefits, depending on the type of material. So, about two years ago, some researchers in Germany conducted a meta analysis. And for those listeners who may not be familiar, that’s sort of a statistical analysis of multiple studies in a given literature. And they did this meta analysis on interleaving studies, a good portion of them. And they found that the interleaving benefits were especially strong for perceptual categories like bird families and things like that, learning to recognize pictures. And whereas for text-based materials, like expository texts and short kind of non-visual materials, there was no substantial interleaving benefits. So I think that sort of raises a cautionary note that we don’t know the extent to which interleaving enhances learning across the wide space of possible things that you could use it for. And actually, that was one of the motivations for our current study, because we were thinking, would it actually work in the domain of physics? That hadn’t been investigated before. And actually, very few studies have been conducted in undergraduate classrooms. And so there was lots of unanswered questions that had not actually been empirically tested that we examined in this particular case. And again, yet other circumstances warrant further investigation before interleaving, I think, could be confidently used.

Rebecca: So we always wrap up by asking: what’s next?

Josh: For us, in the sense of research on interleaving, what’s next is getting at this question of generalizability. How broadly across different types of learning, different contexts, do our results generalize. And we think of tackling this in a couple of ways. The first is, you could tackle this from a theoretical and mechanistic standpoint, meaning if you could understand why interleaving works, and be pretty confident about the factors that lead to why it works, then you can look at it in a given scenario and ask yourself, “Well, do those factors exist in this scenario, and therefore, can we be confident that it’s going to generalize to this scenario?” …so, sort of mechanistic theoretical line to understand its generalizability. And the other option is just to scale. So you could, for example, scale our experiment to something that goes across many disciplines simultaneously. We think both of those are really important and promising directions for future research. So, for us, we think, in the broader scope of interleaving research, that’s kind of a nice thing to investigate next. But also, I just want to say that in terms of just interleaving versus blocking, an interesting questions about interleaving versus blocking, there’s still a question that is largely unanswered, what about some sort of a hybrid schedule? Sometimes we think to ourselves, and I think we actually personally have this intuition, that some degree of blocking at the beginning of a very novice learner’s learning might actually be very beneficial. So, for example, imagine you’re learning to play piano, and your instructor teaches you three different scales, and you’ve never played the scales before, and then asks you to play the scales interleaved. So, play this one scale, then play the next scale, play the next scale. You may just be so non-fluid with playing these scales that perhaps your practice won’t actually be very beneficial, even if it’s interleaved. Maybe you need to play one of the scales two or three times over and then start the interleaving. The same may be true for mathematics learning or for problem solving in physics. So there’s still very much open questions about things like that, like, what exactly is an optimal schedule, and under what conditions do these optimal schedules need to be applied.

Steven: And to add on that, sort of the robustness of the interleaving effect across extended durations, perhaps longer than the four weeks that we sort of implemented interleaving assignments, and then had the surprise exam and different, maybe more advanced, materials, rather than an introductory course, maybe a more advanced level course where the students are already coming in with more pre existing knowledge that’s relevant, and so on. There are lots of possibilities to explore as to the extent to which this sort of interleaving effect generalizes to a variety of different educational circumstances.

Josh: I think talking about scale and understanding how this works across disciplines. I also want to say as sort of a tangential note that I’m a physicist by training, Steven is a cognitive psychologist. We connected in a little bit of a sort of random way at UCLA. But it’s been an incredible thing, having an interdisciplinary collaboration. And I think in our view, such collaborations are just really powerful and useful in the science of learning, in particular, that is becoming this very multidisciplinary field. So on a broader note, I hope what’s next for the science of learning is more of this sort of collaboration across disciplines.

Steven: Yes, and both of us brought unique perspectives and skill sets to this particular project. And it ended up working just marvelously. And hopefully there’ll be more of that in the future.

John: Well, thank you. We really enjoyed reading your study, and I think it’s a really nice contribution to the science of learning.

Josh: Thank you for having us.

Steven: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]