293. Study Like a Champ

The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, Regan Gurung and John Dunlosky join us to discuss a new resource they have created that is designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

Show Notes

  • Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. Routledge.
  • Gurung, Regan and J. Dunlosky (2023). Study Like a Champ: The Psychology-Based Guide to “Grade A” Study Habits. American Psychological Association.
  • Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological science in the public interest, 9(3), 105-119.

Transcript

John: The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, we discuss a new resource designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Regan Gurung and John Dunlosky. Regan is a social psychologist and is an associate vice provost and Executive Director of the Center for Teaching and Learning at Oregon State University, He is the author of over 120 peer-reviewed articles and has co-authored or co-edited 15 books. John is a professor in the Department of Psychological Sciences and the Director of the Science of Learning and Education Center at Kent State University. He co-authored the first textbook on metacognition and has edited several volumes on education. Regan and John are the co-authors of Study Like a Champ, which was published earlier this year by the American Psychological Association. Welcome, John and welcome back, Regan!

Regan: Thank you.

John D.: It’s great to be here.

Rebecca: Today’s teas are:… Regan, are you drinking tea?

Regan: Just water because it is a scorcher here in Oregon. So water, and lots of it.

Rebecca: Hydration is important. How about you, John?

John D.: No tea for me for about five years, I used to really be addicted to oolong. And now I’ve gone full espresso. [LAUGHTER] Water for me today too. It’s a little bit late for espresso on the East Coast.

John: And I have an Irish Breakfast tea today.

Rebecca: Oh, John, you’re really pumping up the caffeine today. Also on the East Coast,

John: My day started at 5:30 this morning…

Rebecca: So did mine.

John: …and it’s going to be going really late. We’re in the midst of grading here, we’re recording this a little bit earlier than it will be released. So we’ve invited you here today to discuss Study like a Champ. What prompted you to create this book?

Regan: So both of us have been teaching for a long time, both of us do research on teaching and learning. And I think both of us really like taking stuff from the lab and testing it in the classroom. And, I think, as passionate teachers, we noticed that we knew the stuff, we knew what students should be doing, we told our students what they should be doing. We even tried to design our classes in that way. But clearly, we needed to do more of it. And the impetus behind this book was let’s put these expert tips in the hands of students everywhere. We didn’t want it to be just those students who had teachers like us who talked about studying, we want it to be in student hands. And that’s where it came from, specifically driven by the fact that even though there’s some really neat stuff out there on learning, very little of that, if any, is written to the student. And that’s something right from the get go when John and I first talked, we agreed that this would be in a voice that would speak directly to students.

John D.: Absolutely. In fact, I think that was the main impetus because of all the wonderful books and volumes out there on learning sciences. And just too many of them, I think, appropriately so written for teachers. And often they tend to be a little bit fact listing because of that. In other words, I’m going to tell teachers everything there is to know about learning, which can be overwhelming. And what Regan and I wanted to do is kind of find the most effective and best little snapshots of the learning sciences to share with students, the things that we think will move the needle the quickest, so to speak. So by no means do we tell students everything there is to know about the learning sciences, but we hope we tell them the best stories to get you back onto the learning track, so to speak.

Rebecca: So you begin your book by discussing a variety of widely believed learning myths. Can you talk about a couple of these myths?

John D.: Sure, one of the biggest and I really don’t think this holds true for college students, but K through 12, definitely, it’s just the myth of learning styles. So, we all have learning preferences. I certainly have preferences. I wish my instructors would sing to me all the time and do my learning through music. It’s just that’s not going to help me learn calculus, and so forth, where really hard work needs to be done. So it’s natural to accept a myth like that, because it makes everyone feel good. It really can also undermine learning because it makes us want to do things that are more aligned with our preferences, but sometimes misaligned with most effective practices. So that’s kind of one learning myth, which I hope, in 10 years, if we were to have this discussion again, it would not come up because it was completely dispelled. But there are a variety of others too, that I think can undermine student success too.

Regan: And I think something else, that we really tried to put ourselves in students’ shoes, and we built on what our students said to us. And another one I wanted to highlight in response to that question is so often I think our students think that learning is up to us, if they have a good teacher they learn. If they have a bad teacher, they won’t learn. And undoubtedly, teacher effectiveness is important, but it’s also up to the student. The student has to do something, the student has to take accountability. And early on in chapter one, we show that at least half of the variance half of the accountability for learning is in the student hands. Teachers are also responsible, but really, if a student isn’t doing their bit, then learning is not going to happen. And now once you are ready to do your bit, well the rest of the book tells you what to do. But you’ve got to realize that don’t just sit back and go “Come on teach. Do your magic.” We all like doing our magic. But we can do all the magic in the world if you are not joining us in this adventure, and I’m thinking adventure a lot, because Zelda Stairs of the Kingdom just got released recently, and my household seems to be quite into it. So yeah, join us on the adventure.

John D.: Absolutely. Let me just jump in with one other learning myth that really gets me, and it literally gets me, because I fall prey to the same thing. That’s if you’re struggling, you’re not learning because it’s frustrating. And I often try to pick up new skills, I’m teaching myself to play the guitar now. And I struggle, and I get frustrated, I want to throw my guitar down because things aren’t happening. And I have to remind myself that sometimes it’s the struggling when you’re doing the most learning. And if we could allow students to realize that if you’re not struggling, you actually may not be learning, and that you should embrace the challenges, the hurdles that are in front of you. And it turns out, some of those hurdles are embracing the most effective learning techniques, which produce struggles in learning, but yet also produce the best long-term outcomes. Whereas other ways to prepare for classes and so forth, the way I’d like to teach myself to play guitar, makes it seem very easy. But then there’s really no progress being made. And if there’s any method I could really want to undermine, that would be a major one, that sometimes struggling is a good thing, not all the time, and it’s up to the students and the instructors to figure out is this a good struggle or bad struggle and how to correct that.

John: Those times when we try to give students desirable difficulties, they often find those to be somewhat undesirable. And I often get comments in my course evaluations that “he’s making us learn it ourselves.” And my response is always “Well, I can’t really learn it for you, I can give you the tools, I can give you resources, I can help you learn this material, but ultimately, you have to do the learning.” Going back just a little bit to your comment, John, about learning styles, I had a discussion in an online class, which is mostly upper-level students. And about three quarters of the students in my class were very strongly convinced of that myth of learning styles. So it hasn’t gone away. We hope it ma, but it is very pervasive, certainly in K through 12, where they’re regularly testing it, but some of the students have mentioned that they’ve been tested here at the college in some of their classes. And it’s again, really frustrating to see that, but it’s pretty pervasive.

Rebecca: Yeah, I was struck too, John, when you were talking about both learning styles and the struggle, because as I was working with mostly graduate students this semester, I had those exact conversations with those students. But it may have been the first time that they’ve really come up with some barriers that they faced in learning, and really had to reassess how they were working towards their goals.

John D.: Absolutely, and nothing but utmost compassion for students and sympathy, because in many ways, the experiences they had before college allowed them to use ineffective strategies and still get by. So things didn’t seem like a struggle, the difficulty now, the first time, then you hit a struggle, when you really have to change strategies or techniques, you may not realize what to do. It might seem frustrating and overwhelming, when in fact, now’s the time when the real learning begins.

Rebecca: Yeah, and I think sometimes when the subject matters new, as well as the strategies are new, students can immediately feel a little cognitive overload. [LAUGHTER]

JOHN D. Absolutely.

John: And part of that I think may be related to how they’re often taught throughout their whole career. In general, much of the advice they get in elementary and secondary schools, and sometimes in college, encourages them to adopt strategies that may not be that effective. Often, people still use high-stakes exams. And people don’t do a lot of interleaved practice or spaced practice, which are things you talk about really nicely in your book. And the incentive structure for students is perhaps to favor things that align well with short-term recall without that longer term learning. And your book can help, if we can get students doing this, but it seems to me like we also have to do a bit more work and getting faculty to adopt some techniques that may encourage students to use techniques that are more effective. Might this be something we could build into our classes a little bit?

Regan: Actually the riff over a couple of different things you said, John, I think there are two things going on here. One is, a lot of the issues that we run into with our students are actually pretty easy to pinpoint. And that is high school. And when I say high school, I’m not saying “Oh, high school teachers,” I’m saying the whole high school environment. One of my favorite examples is the syllabus. Many of us college and university faculty complain that students don’t read the syllabus, and I had a great conversation with my 17 year old. I was talking about the syllabus and he said, “Yeah, the syllabus is useless.” I said, “Excuse me?” and he said, “Here’s what we do with the syllabus. We spend a whole day, a whole day, reading through every word on the syllabus, but the syllabus is not really designed to help us learn.” And I looked at his syllabus. And sure enough, the syllabi he had didn’t really map on to the best practices that many of us listening employ. And I could completely see why he had that whole. “Yeah, I’m not going to read the syllabus when I get to college. I said, “And of course, you will now,” but it was a great conversation where I really got to see the mindset. And I think that’s just one example, where a lot of this happened, where it’s like, let’s see, what have they just been used to? Then add on top of that, how learning changed during remote learning. That’s another whole mix to the whole issue. And then, add one more, the misinterpretation of, I think, learning styles. Before we started this call, we were joking about the fact that our book is now in an audio version. Now, because it’s an audio version, does it mean the auditory learners will be better than those….? No, it just means that sometimes when you want to listen to something that works well, other times reading something works better. So it’s not this whole “Oh, look, there’s this one modality and I’m a person of this one modality.” And I think that preference that John D. so nicely used… it’s preference, right? …and we have preferences. But let’s also not forget the take home message of that Pashler et. al 2008 study, which is the more styles that you use…. And here’s where it goes back to desirable difficulties, mixing it up, is actually better for learning, no matter what your preference may be.

Rebecca: Oh, but mixing it up is so uncomfortable. [LAUGHTER]

John D.: It makes things a lot more difficult, right? One advice I would give too, that my teaching has changed dramatically, as I’ve gotten more into learning sciences. And at least for someone who teaches, I hate to say this, a non-essential course course that I love and courses that I love for undergrads, I realized I was teaching them too much. So I put them into almost a defensive mode, where the only way to prepare, because they were overwhelmed, is to cram as much as they could and hope to pass the exam because it was a tsunami of information. So over the years, I’ve followed the model, less is more, it’s up to me as the instructor to figure out what is really essential, and how to give them just as little as possible to make their experience as large as possible. So that they then can use the techniques to learn the content well enough so that they can keep that content with them a lot longer period of time. So it’s really kind of changing styles. Now, don’t get me wrong, if I’m teaching chemistry to pre meds, and you need to know chemistry, then you need a different approach, because there’s a lot to learn, it’s difficult, and aspects of our book really focus on how to meet those challenges too. And I think students, just like instructors, need to pick and choose where they want their battles, you don’t have to ace everything as a student, but decide what you really want to do well at, and then use the most effective practices to nail it as you work forward toward a long career and lifelong learning.

John: One of the things you include early in your book is a list of what research finds to be effective and what research finds to be ineffective, side by side, which for faculty use could be really helpful except for some degree of similarity between those two lists. Could you just talk a little bit about that.

John D.: I like to think about it, as much as I’ve not talked about it like this, as more or less effective versus effective and ineffective. And I’m not gonna like reference to that table necessarily, but let me talk about one that I badmouth a lot, which is highlighting as a learning strategy. And like Sharpie has not contacted me with a lawsuit or something because it sounds like I don’t want people to have highlighters. And what we mean by less effective as a learning strategy here is, is that using a highlighter is just the beginning of a learning adventure and not the end of it. So there are great uses of a highlighter of rereading material, of these things that really don’t lead to a great deal of learning, because they kind of are the stepping off point or the catapult for learning, so to speak. So is using a highlighter ineffective for learning? Absolutely, it doesn’t help you learn the content that well. But what I recommend to students: to highlight everything they want to learn as kind of a different approach to that strategy, so that they then apply the most effective strategies in learning it. Yes. So as much as we kind of play off ineffective to effective, it’s more like every strategy in its right place. And some strategies that can be generally ineffective for very specific uses might be relatively good, and what I think Regan and I try to do, because it’s way too much to think about if you’re not a learning scientist, like one exactly show us the specialty strategy, but we try to focus on are just strategies that students could learn just more generally to do well in any particular content. So kind of low hanging fruit, so to speak, where you don’t have to be a learning scientist to make decisions about which specialty strategy to use in every place. But here’s some kind of fail-proof strategies that can help you anywhere.

Regan: I want to actually say there’s that flipside too and John, this is something you asked about earlier when you talked about teachers changing what they do, and I think John D, and I joke about the companion volume to this is Teach Like a Champion. But I think the nice thing about that table of high utility and low utility strategies is that it’s not just the student who needs to be aware of the fact that some are high and low. But I think we take it further to say why are some of those high and low, and then here’s the key companion piece, which is, we hope that instructors reading this can also take those tips, and be ready to share different options with the students. Because I know both John and I have compared notes on this, when a student comes to our office after an exam who’s not thrilled with their score, who has not scored well, we both do some pretty intensive. “Alright, so tell me exactly what you do.” Just that conversation is so important. We both ask to see their notes. And of course, every once in a while a student says I don’t take notes. Well, let’s start right there. And that’s why we have a whole chapter on note taking, and why take notes and how to take notes. But I think that’s exactly it, is so many of us. and I’ll say us, not just our students, so many of us are so used to doing things just one way that we’re not ready. And we don’t look enough about is this really working or not. And I think that’s what we really push with a lot of very real examples ripped from the headlines of our lives, as it were, where we talk about, look, this is what somebody’s tried, and here’s what they did wrong. And here’s how they can make it better.

Rebecca: I wanted to talk a little bit about that chapter on notetaking. Faculty often make the observation that students don’t take effective notes, but then don’t necessarily respond by teaching students how to take notes or giving them some strategies to improve that skill. Can you talk a little bit about some of the things that finally might show students or offer students to take more effective notes?

John D.: Absolutely. So the one thing that I would recommend highly is that if we all agree that taking notes in its own right is not where the learning occurs. And in fact, research comparing people who take notes versus who don’t take notes typically showed no differences between the two. Now, the key is, when you look at these studies carefully, you’ll realize the folks who took the notes, and who didn’t take notes do equally poorly. So it’s not like they do equally well. So the notes are a artifact that in many cases students can use to really learn the content well. So the question is, how do you get all the correct and appropriate information into those notes? So one thing that teachers can do, that research shows really helps well, it’s just flag important content, literally tell students “Okay, time to take notes, this is critical, I need you to know this.” One of my favorite statements, ”The next thing I say, will be on the exam.” [LAUGHTER] And see how many people start taking notes. Now, of course, I would love to see notes that are embellished with examples, with questions, and so forth. But at least getting the rudimentary content down there as an artifact, so students have it to utilize as they’re preparing is so important, or having teams of students take notes and share those things as well is also good. So I think for teachers to be compassionate, but also to help your students sometimes identify what’s most important, by speaking louder by saying, “Hey, this is really critical.” Sometimes I have to even flag it to not only just critical, but it’s going to be an exam, you’d think those two things meant the same thing, but sometimes students need a little help. I think it’s important, and there’s nothing wrong with telegraphing what is most important in your class. In fact, let me restate that. There’s everything not right about not telegraphing. Now, try to parse that. But what I’m saying is, we should be telling the students what they need to learn, how we’re going to test them on it, what we’re going to test on, and allow them to meet those successes. And I think good note taking, and helping students take notes by telegraphing what should be in there, can be very useful, in my own experience, and at least some of the research too.

Regan: Notice what else is in there. If we do want students to take good notes, we don’t want to be talking extremely fast. We want to make sure we have pauses, we want to make sure our slides, to the extent we use them, are not so packed with information that either they want to copy it all down or they get it all as a handout so they don’t have to, but there are all these nuances. And I want to go right back to that finding about the students who take notes and the students who don’t take notes. The bigger issue is what are you doing with those notes? I think far often students just think about it as “I’m going to record what the instructor said.” Well, you’ve got to go back and revise them. You’ve got to go back and check and see if your understanding is right. You need to review them. The issue is, I think, when people take notes, they’re not doing enough with those notes. And there are ways where we can talk about how to take better notes and they’re not. When I teach 100, or intro level classes, every once in a while, I will stop, and I do this during the first week, I will stop. And I’ll say, “Alright, here’s a reality check. If you took good notes, you should have the following. How many of you have it in your notes?” And I do that a couple of different times that first week or the first two weeks to get into that habit of you should be taking notes, you should be doing something about it. I also want to say something that didn’t make it into that chapter on note taking, but it’s gone up another shot, because I’ve seen this so often, in the last year and a half or two years, students taking photographs of the slides. Reality check, people, that’s not taking notes, taking a photograph of a slide is not taking notes. But I think in some students’ minds they’ve taken notes, just more effectively. And there are studies now rolling out that show that the photograph slides versus note taking versus controlled no notes. that the photographs and no notetaking is doing the worst on quizzes. But I think that’s something else we’ve got to take up” how you take notes is important and taking a photograph is not it.

John D.: Let me spin back to something that Regan said that I think is great, too, as far as helping students develop better notes, I begin, and I know Regan does this as well, every class with a no-stakes quiz. And I use that for a variety of different reasons, some of which we talk about in our book, but one of them is first try to answer the quiz multiple choice question without the alternatives from memory. And if you got it from memory, that’s important, because that’s going to help you learn. And then if you don’t get it from memory, now see if you can answer using your notes from the last class. And if they can’t answer the question using their notes, we have issues. So they’re not taking complete notes, they don’t understand their notes, or the issue could be with me. Occasionally, I look at the notes and realize, “Oh, my goodness, I must have said something wrong last class, because everybody’s notes are incorrect.” [LAUGHTER] And if everyone’s notes are incorrect, the source of the error is probably me. So it’s a way to help students understand that the notes are the vehicle to understand the content that we really want them to learn. And there’s ways to reinforce that through no-stakes quizzes, and so forth to help them understand that what we’re doing in the classroom is providing all the scaffolding to help them succeed.

John: One of the things you emphasize early in the book, and throughout the book is the importance of students developing their metacognition as well as their self-regulation skills. Could you talk a little bit about why that’s important, because I think that’s something that’s not emphasized enough throughout their educational experience,

John D.: I’m gonna just focus on one aspect of our particular viewpoint on self regulation, and then I’ll let Regan go with the more fun stuff. I want students to really succeed in school and in college. But quite frankly, as a college educator, I want students to succeed in life after they leave here. And one of the most important skills that anyone can learn is time management, that’s about self regulation. And believe it or not, I have actually students tell me, it’s like, “Well, I know, managing my time and developing plans, that’s just not my cup of tea.” (speaking of teaching for tea). But it absolutely should be, because effective people develop goals, plan on how to get them, and manage their time to get there. So if students can take something from this book that I think is highly valuable about self regulation is to look at our examples and our encouragement and inspiration to manage your time, make goals and plans, and develop a time management strategy that works well for you. Because that’s ultimately going to lay the foundations for a successful use of other strategies we discuss in the book, but also for a highly successful life, whether it be success in your job, your hobbies, raising a family, or whatever, successful people manage their time and self regulate. That’s something that we really do push in the book, and some of it’s not necessarily metacognition, per se, but it’s about really taking ownership of your life and deciding how to get things done effectively. So time management, for me is so important aspect of this book.

Regan: I think whenever you think about self regulation, I really try to stress two elements of it on one hand, especially think about metacognition is the: “What do you know? Do you know this stuff?” And there’s that whole “Am I ready for this exam? Do I really understand this concept? Do I know how to do this?” That’s the classic metacognition, but even more broadly, when we talk about plan and monitor and assess your knowledge, that’s, I think, really important as well. And of course, the tips such as retrieval practice is great for the assessment too and to assess yourself. But I think I love the way John talks about time management because when we talk about self regulation, a big issue that I hear from students a lot is where those two things interact, not being able to regulate how they use their time, especially when it comes to social media. And students will say to me, “Yep, I start scrolling Tik Tok, and before I know it’s an hour later. Well, that’s a basic self-regulation issue. That’s a time management issue. That’s a planning issue. And I think in our planning chapter, we recognize this and we tell students “Look, it’s okay to enjoy social media. But guess what? Plan your social media timr. So it’s not like you’re saying, I’m not going to use social media. Be real. If that’s what you enjoy, sure, look at some social media, but plan it out, allocate time for your social media, just like you allocate time for your work so that everything fits into your schedule.”

Rebecca: Allocating time, it feels like have a lot of conversations with students about that on a regular basis.

John D.: Oh yeah.

John: Are there other topics you’d like to emphasize about your book that we haven’t touched on yet?

Regan: Well, I think one thing that if you’ll notice, the last chapter in our book is all about things that, for the longest time, nobody thought related to studying: sleeping, eating, physical activity. But I think that is just so important. That is so important. And gosh, I know, I don’t think I ever shared this with John. But sometimes I almost played with putting that chapter first. We did put it at the end, because I think it’s really important. But I think most students don’t realize how those life things interact. And I think for those of us who read up on higher education, there’s just so much written about burnout among students, among faculty, and all of that relates to those things we talk about in the last chapter. It’s prioritizing sleep and prioritizing good eating and prioritizing physical activity, prioritizing just getting out and getting some air. And some of my favorite comments that I get back from students… it happens a lot when I teach health psych… but even otherwise, where they will say, “you know, yeah, I caught myself scrolling, I went outside for a walk, and it was just five minutes, but I came back and I could tackle things better.” And I think, people, it’s 360 degrees of living is what we’ve got to practice.

John D.: And I love that Regan. Because obviously, we saved the best for last with that chapter. But, come on, to do all these things. It’s a time-management issue. If I’m going to work, all of this in, how can I do so so I have enough time for play and I have enough time for work, which are two important aspects of life. So, totally agree.

Rebecca: Well, they always wrap up by asking: what’s next? [LAUGHTER]

John D.: Wow, that’s a big question. And what’s next, I would say for me, the strategies that I think are most effective, that work the best, unfortunately, have also been investigated the least, because they’re difficult to investigate, they involve multiple sessions for students to engage the same material, and so forth. So as someone who was kind of born and raised in a laboratory, either my own lab or laboratory of the classroom, what I’d love to see is just much more evidence-based research focused on further understanding, not only what works best, but how students can engage in the best practice in the most efficient way. So it’s balancing both an understanding of wanting students to obtain their learning objectives, but also understanding that they want to do so effectively and efficiently. Because there’s so many aspects to life beyond just school. So I’m always going to say more research, although I could totally understand some would say more application too, but I’m gonna go what’s next is some more research and all these really effective techniques and how to use them more effectively.

Regan: And it’s that last part, John, that is what’s exciting me right now, I’ve been learning a lot more about the whole field of implementation science, which is just because something is effective, and something is efficacious, what are the factors that influence its implementation? …because whether it’s a student reading our book and trying to implement our recommendations, or a faculty member, instructor, teacher listening to us and trying to implement what we’re saying, there are still nuances, there are still contextual factors. And I think we’re just getting better at intentionally and systematically teasing apart what some of those issues with implementation are. And for me, that’s been pretty exciting. I was fortunate to read a lot of a fellow Oregon State University, new PhD, Dr. Rachel Schweitzer, who’s done a lot of this work on implementation science, and some of her ideas. I’d love to be able to test them in my class, to dovetail with continuing research on study techniques and how to make it happen more,

Rebecca: Some really important stuff that needs to happen, for sure.

John: And thank you for this book and all the other research and work that you’ve done in support of improving student learning.

John D.: Thanks for having us on.

Regan: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

282. Moving the Needle

The study techniques that most college students adopt do not align with what research tells us about how we learn. In this episode, Sheela Vermu and Adrienne Williams join us to discuss what happens when an instructor in a community college biology class attempts to encourage students to adopt evidence-based study methods. Sheela is a biologist at Waubonsee Community College in Sugar Grove. Illinois. Adrienne is a biologist at the University of California, Irvine.

Show Notes

Transcript

John: The study techniques that most college students adopt do not align with what research tells us about how we learn. In this episode, we discuss what happened when an instructor in a community college biology class attempts to encourage students to adopt evidence-based study methods.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Sheela Vermu and Adrienne Williams. Sheela is a biologist at Waubonsee Community College in Sugar Grove. Illinois. Adrienne is a biologist at the University of California, Irvine. They are co-authors of a study entitled “Moving the Needle: Evidence of an Effective Study Strategy Intervention in a Community College Biology Course.” Welcome, Sheela and Adrienne.

Sheela: Thank you.

Adrienne: Thank you.

John: Today’s teas are:… Are you either of you drinking tea?

Sheela: Yes, I am.

Adrienne: We heard that was a thing.

John: And what type of tea?

Sheela: I’m actually drinking my favorite tea. It’s called A Sama tea. It’s a calm relaxed, lavender, rose, chamomile and cardamom.

Rebecca: You just describing it just took my blood pressure down. It sounds very relaxing. [LAUGHTER]

Sheela: I was doing it for myself. [LAUGHTER]

Rebecca: Well, you helped me too. [LAUGHTER]

Adrienne: I can’t compete with that. But I have a classic Trader Joe’s pomegranate white tea.

John: Very nice.

Rebecca: I think, John, today was my closest call of not having tea. Because when I got home to record about an hour ago, we had no water at my house. The water was off, but it’s on now. And I have a Scottish afternoon tea.

John: And I have a black raspberry green tea, which I haven’t had for a while but it’s nice.

Rebecca: Yeah, that is one that you haven’t had in a while.

John: I had it last time when I was in quarantine at home, [LAUGHTER] which is where I am now with COVID.

Rebecca: It’s like you’ve established quarantine routines. [LAUGHTER]

John: Unfortunately, yes, this is my second time. We’ve invited you here today to discuss the study that you both worked on. One of the things that you noted in the study itself is that there’s relatively few studies of student study strategies at the community college level. Even though community colleges provide an introductory college experience to over half of all the students that graduate with a bachelor’s degree in STEM fields. Could you give us a brief overview of your study?

Sheela: Thank you, John, for asking that question. Because it sort of made me think about what really brought about my interest in studying this topic. It stemmed from me becoming a community college Bio Insights Fellow, and Insights is a network of community college instructors who are actually interested in investigating teaching and learning in their classroom. So when I became a fellow in that network, one of the goals was for us to think about what are some teaching practices that could inform the scholarship of teaching and learning in a community college setting. And I teach microbiology and anatomy and physiology at Waubonsee Community College. And while I was starting to teach, I realized that students really struggled to study effectively. And community colleges occupy a very important position in higher ed, especially in STEM because they provide low cost training and education for workforce training, preparation for transfer, and also recently an opportunity to reskill for many of our underemployed and our underserved students and population. So keeping that in mind and our classroom structure, we noticed that in the biology education field, papers or authorships for community college faculty, in a CC context was very few, only 1- 3% of all the biology education research articles had a community college context question or a community college. So this network sort of enriched me to think about what do I need to ask a question in the classroom that would encourage my students to use better study strategies? So in some ways, I wanted to ask this question, but it was the Insights that helped me think about this from the scholarship of teaching purposes. So the brief overview of what we see as the basic study of this paper, is we were really wanting to ask this question, what kind of study strategies students are using in a community college context right now in a biology classroom that I teach, that was one… and can an intervention of some kind from an instructor really intentionally encourage students to reflect upon their current study strategies and guide them in some ways to change their strategies to ones that have been shown through research to have high impact. So it was to just gauge the field, but also to see if we can gently intentionally guide that providing guiding practices. And that’s sort of the big picture of what we did. But we administered a pre- and a post- survey. The pre- and post- survey was taken from a previously published work. The survey actually asked questions about the study habits and the study strategies that students actually used. The intervention was administered. And one of the things that is really important is I was able to get approval from our college institutional effectiveness team, and those are the ones that serve as the institutional review board and they look at the studies research paper and also helped assist with gathering institutional data, because institutional data was very meaningful to our study. And Adrienne helped arrange the statistical power. That is what made a big difference in the study. And she also helped me wrap it up and write it up and get it published in the CBE Life Science Education journal.

Rebecca: So a lot of students enter college and plan to acquire a STEM degree but often change their mind or change their plans. Which students are disproportionately likely to give up on their planned STEM degree.

Adrienne: In general, regardless of institutions, students have a difficult time with a STEM degree. They tend to give lower grades and that can be very discouraging for students, even if they’ve done well in high school… when they get to college, and they start to not get A’s can be very discouraging. So there is a history for all undergraduates, when they enter even a four-year or a two-year program that they start to leave STEM. They find it less rewarding than they had thought, I would say community colleges have additional hurdles to overcome, because they are an open access system. And so they get students with a wide variety of past experiences. Some are people who’ve just come out of high school and are used to studying and are pretty on top of things… they remember their math, they’re accustomed to memorizing things. And then there are students that are adult learners, or perhaps have families, been in the military, perhaps they get a GED and have worked for awhile. And so there’s a lot of habits to relearn. That can cause problems, particularly in STEM where the grading is just historically rather harsh.

Sheela: I would agree with what you said about historically STEM attrition is pretty problematic. But the problem is also more exaggerated in marginalized and minoritized students who come from backgrounds that could have been a first generation, some kinds of a financial issues, students who probably did not have a whole lot of high school curriculum that prepared them well for a STEM field. And there’s been recent work in 2016, that talked about how the National Academies needed to look at improving underrepresented minority students persistence in STEM, not just entry into STEM, but for them to be able to stay in the pipeline, and successfully move on and build a career trajectory for themselves.

John: We saw similar results in an earlier podcast interview with Peter Arcidiacono from Duke. He was a co-author of a study that looked at the determinants of students’ continuation in STEM fields and found something very similar: that, holding other factors constant, females and students from historically minoritized groups were much more likely to change out of the STEM fields than students who were white males, even when they were doing relatively better in these classes, then the students who chose to remain in the discipline,

Adrienne: STEM classes, particularly, are not welcoming to many students, because the exams are difficult. Now, there’s just the culture that we’ve developed, not necessarily for good reasons but it does cause many students who are doing fine, who are scoring above the mean in the class, to feel like they’re not succeeding, because they’re only getting 70-80%. And that’s just kind of an unfortunate reality that we’re working to change. But in the meantime, we would like students to be as successful as possible on their exams.

Sheela: And recently, there’s been some studies that talk about some concrete steps to diversify the scientific workforce. And that came out last year in Science, and that talked about how students in the STEM field sometimes do get discouraged, and often feel really compartmentalized by the climate and some of the teaching methods and assessment, exactly what Adrienne was talking about in the STEM classroom. And there’s been an exodus of students, specifically from the underrepresented minority communities.

Rebecca: Can you talk a little bit about some of the study techniques that students were using before your intervention as part of your study?

Sheela: Yeah, so when I walked into the classroom, what I noticed were that students were typically cramming the night before the exam, or perhaps two days before the exam, and they would use one study session to just get all the notes that they could for that particular unit. The other thing that I noticed students doing were, they would be using a lot of flashcards, they would walk into the classroom with a whole bag of flashcards and sometimes it would all drop and they would have color-coded flashcards, different colored highlighters. And as the classes were going on, as I would see them studying, I would see them using most of these methods which was underlining key words in the textbook or terms or in the PowerPoint and in In some ways, creating some flashcards, and massing all of this study as they thought it was in one particular session, and then just go to the exams. And I saw them a little bit frustrated thinking that they had put in so many hours into it, and not having the results that they would have expected out of those exams.

John: Could you tell us a little bit about what study techniques you recommended to students and how you convey that information to students?

Sheela: So one of the key things that I did for myself was just to get familiarized with what were some high-impact study strategies, because this was not something that I was very familiar with. And one of the things I figured out was it was distributed spacing. And in the STEM fields, especially in the subjects that I teach, like microbiology, anatomy, and physiology, we wanted students to know one concept well, before they moved on to the second concept. So if they tried to do all of this studying the night before the exam, it’s very possible that they missed out one of the pieces in the Jenga, and the whole Jenga fell apart the day of the exam. So, something that I talked about was spacing, but also making sure that spacing is not just happening two days before, but it is distributed throughout the unit that we are actually covering. And the second high-impact practice that I asked students to think about was self testing. Self testing sometimes can be difficult for students to understand, especially at our community college setting, because the kinds of self testing that they’ve been exposed to in high school, is a teacher giving them a set of practice problems, or they having 50-60 questions in a test bank, and just going over those, and in some ways, just memorizing or flash carding and making it into a Quizlet. And they would think that is self testing. So I had to go back and walk them around and say that self testing would be to self test a concept. And to double check and see, do you really understand that concept by looking at a problem or a question, which is at the end of the chapter, before going on into the next particular concept? The other piece that is also very important that high impact practices that I talked about to the students was diagramming. And this was something that I wanted them to get away from, just trying to just reread chapters, condensing chapters, and summarizing chapters. I wanted them to get away from those kinds of practices and move on to something called diagramming.

John: So what did you find in terms of the use of self testing and spaced practice, as well as the drawing applications? Was there a significant change in the student use of these techniques over the course of the semester?

Sheela: So, in the beginning, there was some resistance from the students. Because this was something difficult for them to do, they had never heard of these kinds of strategies. And as we proceeded with the intervention, which was periodic, we found that students, over the period of the entire semester, improved their spacing, so they went from high levels of cramming to high levels of spacing. We also found that self testing, which was very low in the beginning, also improved at the end of the course. Students were not creating their own questions, but instead, they were using the questions that were at the end of the textbook that were desirable for comprehension. They were using questions that were problem-solving oriented, which were part of the assessment to test themselves and diagramming also improved by the time the semester finished.

Adrienne: Let me also say something that we noticed was students were willing to give up some strategies very easily. I do similar work at my large four-year R1 institution, and students also arrive in their first quarter at UC Irvine and they’re very fond of flashcards, and tend to also be fond of underlining and highlighting. And those are two study strategies that students see kind of quickly just don’t work. The exam questions that they get in college are just not associated with things that they did on flashcards and so they’re like, “Okay, boom,” and they drop them with very little resistance. Every year I check, every year, they come in high flashcards and they leave low flashcards. They’re like, “Nope, that didn’t work.” And they’re happy to drop that. Other study strategies like rereading their notes: they come in high rereading their notes, and we tell them, that won’t help. Yes, you have to understand the concepts, but stop rereading your notes, that is not an effective study strategy. And that one doesn’t move. They’re like, “Nope, still gotta reread my notes.” And so that’s been interesting for us. I talked to Sheela about that. I talked to students about that. And so some of these strategies are easier to move than others. And I think what’s going on… we’re going to foreshadow some metacognition here in students choosing what study strategies to use. They feel like when they start their study session, they have to decide what is it that I’m going to work on and so they reread their notes as a way to set themselves up with what needs to happen in the study strategy time they have coming up, but it just takes so long to reread all their notes that they know half their times gone by the time they finished doing the setup activity. And so they’re unable to do the more useful things of self testing, explaining to others, doing things from memory, painful things. It’s also just more encouraging and soothing to look at your notes and go “Yep, yep, I remember that. Yep. Okay, yep, I remember that.” And that feels like studying, even though not really.

John: That’s sometimes referred to as fluency illusion, that the notes are familiar, you see them, you remember the organization on the page, and it’s reassuring, even though it adds virtually nothing to your ability to either recall concepts or to transfer those concepts into different applications. But it’s hard for people to give that up, especially when they’ve been told to reread their notes and reread the materials over and over again, throughout all their prior educational experience.

Adrienne: Yeah. And it worked great for him in high school. So why would they believe us until they try it and find out that it doesn’t work? So some strategies are easy.

Rebecca: So much easier when your answers are in your notes, and not from learning?

Adrienne: Well, the ironic thing is, in my classes, I’m now all open note, open Google. And so they still think the notes will be the answer to all their problems. Now, it’s all application problems.and so you can’t find all that. So it’s hard, it’s difficult for students to change, even if they know like, one of the new things I’ve started to ask is, what do you think researchers say is more effective: spacing your studying or cramming? And all the students know that spacing is more effective. And then you ask, and what did you do for this test? And they’re like, “Well, I had to cram.” So a lot of it is not lack of understanding of what should happen, but just the difficulties associated with it, just like eating right and exercise, the difficulties with actually doing the difficult work.

Rebecca: I think often students mentioned that it’s difficult to plan their time, or to have the time to do the thing that they know is good for them.

Adrienne: Which is why we’re going to again, come back to metacognition, I think, that this talk about how we’re kind of moving forward out of just asking students, how are they studying, but thinking more broadly about metacognition.

John: Before we get to that, though, could faculty reduce the incentive to cram by using more low-stakes activities so that students don’t have that incentive to cram before a high-stakes exam and ignore studying the rest of the time.

Sheela: So John, in the courses that I am teaching, we have not just one high-stakes exam, we have many small unit exams. And these unit exams come every two to three weeks. But the material in STEM, as we speak, is so dense for the students that they have to move from lower order Bloom’s… just remembering terminology… all the way to concept analysis to an explanation very quickly, in a very brief period of time. And I think even having those low-stakes assignments is not enough for them. Because those assignments, they may not choose to use the high impact study strategies, they may get away by looking at a summary, or maybe looking at the notes in the low stakes. But when they come to these unit exams, even if it’s just two chapters, I found students in my college, in my classroom, really struggling, even if the low-stakes assignments were done at a 95% completion, which is what led me to think of this study and say, what is it that I can do as an instructor, I would love to change all the dynamics of higher ed, and move things seamlessly to make it a beautiful world for everyone. But I can only control what I can do in the classroom, which is the intervention. And I think a persuasive intervention through modular use over a period of time, which is consistent, and short and brief, is a possibility that faculty could use in order to shift students’ practices of study strategies. And for them to be cognitively aware that this is a good study strategy. I’m actually aware of it. And that’s the knowledge I have from the literature, which is what Adrienne was talking about. But then how do you implement it in the moment and modify just a little bit so that you can actually get good grades in that unit exam that’s coming? Because there are five of them, you just can’t afford to blow each one of them. You have to just get gooder and gooder if there is a word like that. [LAUGHTER]

John: You mentioned working to improve students’ metacognitive skills. Could you talk a little bit about how you built in metacognition into this approach.

Sheela: So when we did the intervention, my intervention was very simple. It was a PowerPoint presentation, and it was a PowerPoint presentation before and after each exam. And we have five big exams in that course. And we had an exam wrapper as well. So the PowerPoint presentation was not just this is what you need to do. It was things from the literature of high-impact study strategies, and also being aware of what is a low-impact study strategy. And the PowerPoint presentation was 15 minutes long, it had about 15 to 16 slides. But there were some examples of how to use those strategies and how not to use them. For example, if they were looking at muscle contraction, and if they were looking at how the skeletal muscles, students know that it’s a bicep, how does the bicep actually contract instead of just making 100 flashcards of every piece of that information, which is in that unit? How can we translate that into concept, and learn each concept and see how that moves into the next concept. For example, when we think about this, you think of a motor neuron that actually stimulates a skeletal muscle like a bicep, and there are multiple segments in this piece, the neuron has to send a signal, which is an action potential, so they need to know what’s an action potential. And they also need to know the structure of the neuron. So there are two big pieces there, then it sort of travels into a terminal, and it causes some channels to open, they need to know the kind of ions that actually travel through those channels, and what causes those ions to travel from A to B. And then through that influx of those ions, they need to know there is a release mechanism. And that release mechanism causes another set of ions to open postsynaptically in the muscle. And from there, they move on into understanding how it contracts the muscle tissue or cell. So there’s a sequential activity that goes on. And for students to be able to compartmentalize that and get good at understanding each concept before they move on to the next is something that my intervention was part of. And for them not to just make all of that into a highlighting flashcard, summarizing it into three sentences and say, the neuron travels and the bicep contracts. And in the middle, there is a gap. And that’s called a synapse. And that’s what they would do when they summarize. And that’s not effective, because the questions are not summary driven. The questions are application driven. What would you do when a drug blocks that particular channel? What would happen upstream? What would happen downstream? And now they’re like, “I never thought about this much detail.” So that was the intervention. But then there was also an exam wrapper. The exam wrapper, it was more of a reflective piece. I just thought it would give students an opportunity after every exam to self reflect and see what study strategies they used. How many hours did they spend studying? What would they like to change for the next exam? And I thought it was reflective, I gathered a lot of paper and I gathered a lot of data, it’s sort of gave a quiet moment for our students to reflect on exams. But talking with Adrienne and working on this for a little while, realized that the research on exam wrappers does not show… it’s not efficacious enough to change students on how they learn. So now I’m doing something a little bit different where I’m not just doing an exam wrapper and calling it as a check in point on Canvas for them to sort of reflect after the exam, but not just using exam as the main tool, which is what shifted us to think about the next steps, which is the metacognition, which Adrienne was talking about.

Rebecca: Adrienne, do you want to share some of the details about the metacognition side of it here?

Adrienne: Yeah, I do research on my students. And it’s just been fascinating how students seemed to choose to do strategies which didn’t seem helpful. And so in additional reading that we’ve done, Sheela and I have learned more about metacognition. And you can kind of break down metacognition, in knowing what good strategies are… like, do students know that spacing and retrieval practice and interleaving? And diagramming? Do they know that those are considered the good strategies? Second, do they have the metacognitive skills to use them at the right places, or even though they know spacing is good, they never actually set up time to do the studying days in advance. And then thirdly, metacognitive judgment, do they have a sense for yes, they now know this information, can they judge their learning? Is their appropriate judgment of learning that goes on? And so all of these are steps that we realized, all we knew was which strategy students chose. We didn’t know if they had the knowledge about which ones were good ones. And we didn’t know whether or not they recognize whether or not they were successful. And so we’ve expanded our questions to students: do you know which strategies are successful? …and pretty much they do. That doesn’t seem to be the missing link. It’s the scraping together, the organization time to actually apply them early enough that they can spend the time appropriately. And we’re also trying to determine can we help students? Give them regular feedback? The exam wrappers didn’t seem to work really well. There wasn’t a lot of evidence that they were really doing a lot of changing of student behavior. So, something I’m trying this quarter, particularly in my anatomy class, is to ask students: “If I said different study strategies were worth different amounts. I said, rereading your notes is only worth 0.2 points per hour and I said, explaining concepts to a neighbor or drawing diagrams by hand of the different systems and that’s worth 1.5 points per hour, will you change how you study in order to maximize your studying points? And so we’re in the middle of that right now to see is there anything that can motivate students to attempt to try new, difficult, painful, complicated things, other than the comforting things of rereading and rewatching videos? That’s a bigger metacog picture I’m working on, Sheela’s also doing that, kind of increasing our focus away from just single study strategies to this metacognitive view, do students have the metacognitive chops? And is there anything that we can do as instructors to help them with these applications? Because once you’ve told them that spacing is important, they’re like, ”Alright, spacing is important.” But that’s not solving the problem much. It’s an important initial first step, but are they actually going to figure out how can I schedule an hour a night on my anatomy, so that I have some chances to forget and relearning, and so that relearning really kind of nails down those mental pathways, and so it’s easier for them to do the work for application problems when they get to an exam? So we’re enjoying just trying different things as we move forward, to see can we expand our understanding of metacognition. help students understand? Can they tell us what they understand and don’t understand? And can they tell us what they really struggle with in the application so that we can help reward them for doing good studying? Happy to do that.

Rebecca: Have you tried sharing a study plan for the week just to see if they followed it to see if it worked?

Sheela: Yes. And I think that’s sort of the third part of metacognition, that Adrienne was just talking about. The knowledge is one, which is the strategies that they know, and whether they can actually apply those strategies when they are thinking or listening or reading difficult tasks. That’s very important, because you can apply these strategies when it’s in your comfort zone. It is the application of these high-impact strategies when you’re out of your comfort zone, and that is really important. And that’s where the planning comes in: having a calendar and making sure you plan it, and you’re putting in the hours. But what I found in my explanations with my students is that even though they planned the time, I had students who had devoted an X number of hours, they just didn’t know when they got to the muddiest point that they were even muddled. [LAUGHTER] And that is, to me, metacognitive judgment, right? You know, that you don’t know, and you know, that you need to do something that you don’t know, and students were like, they didn’t even know they were muddled. And so they didn’t go ahead and use the appropriate strategies, or change directions, or make some adjustments one week before the exam so that they can actually monitor their learning, which is where we are working on building the set of skills, and just making sure we ask our students and see if we can shift their practices to more of a judgment outside of just planning the time. Planning the time is very important, but it’s not just planning the time. What do you do with that time? And we couldn’t be with them all the time, to sort of shift it.

Adrienne: You’re probably familiar with the idea of high-structure courses, having many assessments. And I think that is really important. It’s a lot more work for the instructor to be building all of these assessments and managing them. But the more cases where you can help students get feedback on whether or not they’re learning successfully, the more likely they’re going to realize that the thing they thought would be fine is not actually working.

John: You talked a bit about ways in which we might be able to better improve student’s understanding of effective study strategies. But did you find any impacts of the techniques that were used during this experiment? Did it make a difference for some of the students?

Sheela: The drawing did. To me, the way I was thinking of drawing was, I was thinking of it as a visual representation of a science process. That’s how I was thinking of it. But the drawing was very meaningful to the students, because students before in my class, were sketching typically, or taking a diagram or a figure from a textbook. Because most science textbooks have beautiful colored large figures, they will just take that figure and translate it into a sketch and just draw some diagrams and draw some arrows and point to some facts. But drawing the process was very important. And not only drawing the process as a flowchart, but actually organizing the conceptual information and connecting the dots. So in some ways, the way I was using drawing representation was more like a concept mapping. But I also realized that students without a lot of encouragement on how to do these concept maps or how to do these representational drawings, were not getting a lot of feedback, because they were just drawing and maybe they’re drawing it beautifully. But they were not really using that drawing to really understand and self test themselves on some of the key concepts. So this semester, what I’m actually doing is asking for our students to show some of their drawings and upload it on Canvas. And I have two or three criteria before it’s accepted as a drawing. And one would be to make sure that they have some basic notes that they have talked about, some key concepts. And then they have also asked themselves a question, which would be like a feedback, like a retrieval question using that drawing. So they’re using the drawing as a practice. And they’re using the drawing as a self-retrieval practice to ask a question. And I’m hoping that it would have some change. But the literature in the drawing area is not very clear. And I would like to use it as an if and then statement, maybe, like, you know, if this happens to the sodium, and what happens to the action potential, or maybe like a causal effect kind of statement, or sometimes maybe even say, why and how, why does this happen? And how do you think you can make it better?

Adrienne: It really does take time and scaffolding for students to be effective at something like this. You can’t just assign them a drawing or tell students you should draw more when you study. But taking the time in class to assign something, give them appropriate scaffolding, give them feedback, show them what you were imagining it would look like, asking them a test question that should have been easy to answer if the diagram had been appropriately written. And just training students, we tend to do a lot of assumption that somebody before us did the training, and that’s just not appropriate in many cases. It is really helpful to make space during class time, either by flipping the class or by flipping 10 minutes of the class, moving that outside to pre-class work so that you have time in class to train students how to think carefully and study effectively so they can be more successful going forward.

Rebecca: I think one of the themes that I’m hearing both of you point to is helping students prioritize things, there might be a sequence to knowing or what might be most important versus kind of an extraneous detail that’s not as important until you have the big thing figured out. And a lot of times when you’re new to something, these are not obvious things. But when we model how to make a diagram and verbalize how we made a decision, or how we chose what might go in a diagram, it can be really helpful and enlightening to students because they’re seeing how that thought process might work. If they’ve not experienced it for themselves.

Adrienne: It’s not like students aren’t working hard. I think we tend to think that somehow students that are doing badly in the class are slacking, and that is often not the case, they’re working very hard. But if you’re spending hours making flashcards, that’s just not as useful a thing to spend time on

Rebecca: …for hours going down a rabbit hole that’s like not actually important… [LAUGHTER] which I sometimes have had conversations with my students like “You spent how long on that? Yeah, that’s not something to worry about. Maybe you should do this other thing instead. [LAUGHTER]”

Sheela: Yeah, and color coding. And our textbooks in the sciences have a lot of colors. And the textbooks also have a lot of highlights. So in some ways, I feel that some of these high-impact study strategies are not very, very clearly explained in our textbooks. So when they see the highlighted word in a textbook, or they see the color-coded diagrams, the students often believe that that’s the secret. In one of our exams recently, I asked them to bring that color-coded diagram and see if that helps them answer the questions. And they were surprised that they could only answer 50% of the questions on the exam with that color-coded diagram. The rest had to be some kind of retrieval practice, some kind of higher-order thinking which they had not spent the time doing, because there was spending more time drawing that thing out, like a sketch.

Rebecca: …and probably essentially just copying whatever they saw. [LAUGHTER]

Sheela: and just making it look prettier. That’s it.

Rebecca: Yeah. [LAUGHTER]

Adrienne: The good strategies are painful, they’re difficult and painful, and expose everything you don’t know. And they feel like you don’t understand things if you’re doing the good work. And that’s a difficult thing to fight through. Nobody likes to feel like they don’t understand.

John: I think we all face that challenge with our students.

Adrienne: Um hmm.

John: They like strategies where they could sit and passively listen to someone explain concepts to them. But that doesn’t really seem to help them remember it, but it feels good. It feels like they’re understanding every step of it. But when you’re trying something in active learning, and you’re getting feedback that tells you that “No, you don’t quite have that,” it’s certainly not as pleasant. And there’s no real way around that, other than perhaps reassuring students that that’s an important part of learning. I’ve been trying to explain that to students for years without complete success, because they much prefer to listen to a lecture or to read a book and assume that they know things until the professor tricks them with these questions that clearly are not a fair measure of their learning, and it’s a difficult cycle to break.

Adrienne: True for all humans. We like to minimize the calories we use in everything.

Rebecca: Is this a line of research you’re hoping to continue on?

Sheela: Yes, we are, as we speak, to fine tune what we learned from the study last year. By the time the study finishes and the time the paper comes out a whole long time has gone by. So that was a great time to sort of think about what are some areas and it was the exam wrappers that sort of prompted me to think about this metacognition, especially metacognitive judgment, because exam wrappers are self reported, they’re sort of reflective, and I found that students were reflecting the same thing, like I spent only 5%, doing X, it hasn’t changed from unit exam to the next unit exam. Then the question became, “Why is it that they were not able to monitor their studying, or make those adjustments as they moved forward?” So maybe for an exam on tissues, they spent 10 hours studying a certain way, but probably when they come to the nervous system, they probably need to modify that or adjust it, but students were not able to do that, based on the difficulty of the task. They were just steamrolling it. They were just doing the same thing over and over again, they said, Oh, “You told me to study X number of blocks, and you told me to study 12 hours and I’m just going to do those 12.” And that’s prompted us to think about the next steps. And we are asking this question in a community college classroom and see what kind of metacognitive skills students have. And we are dividing those skills into three parts, which is knowledge of the skill, are you aware of it, can you actually apply it, judgment would be can you apply it and monitor and change it when needed based on what’s happening in the moment, and then sort of plan and have a control on your self-regulated strategies? I think this is an uphill task, because the data that you get could be a little messy, just like most education research, but I think we’ll just have to continue and plod through it just like how we did the other one, before we get some kind of a baseline that suggests that whatever we are doing, whatever intervention we are planning to have in the classroom, has an effect. So I think of myself as a practitioner, Adrienne is also a practitioner, but she’s also a researcher. So for me, if it doesn’t make sense in the classroom, I’m very happy that it did make awesome sense in the world of research, but I would like it to really make sense in the classroom because I want to see our students benefit, move forward and have great STEM careers, however it may be.

Adrienne: I’m also doing projects. I’m continuing to work with Sheela on adding some metacognitive aspects to her class. A couple of things that I am trying to work on is an implementation strategy where I try to have students, each week, think back on the week that they’ve just completed, and how did they study on that week and so just regularly get feedback and to overtly tell them each week: these are high-impact strategies, these are low-impact, which ones did you use and what do you plan to do next? And a lot of hitting over the head, perhaps. But it takes a lot to change this for them. And secondly, I’m particularly interested in students who are studying with friends. I saw an interesting effect in my intro bio class a year ago, where students who really valued studying with friends were doing worse in the class. And so I’m attempting to figure out what’s going on there, because that was kind of non-intuitive to me. And I think it has something to do with some students really value studying with friends, because it’s an opportunity to have their friends explain the material to them. So what I’ve been asking students this past year was what do you do when you study with friends, they’re like, figure out what they’re actually spending their time doing. And some students are spending a lot of times explaining and others are spending time getting explained to and there’s different relationships in how they do in the course. That is pretty clear, though students that are in study groups in order to learn the material, that’s an indication they’ve got other struggles going on. They need help. So things like that, I’m still figuring out the best way to ask students and to figure out where the pain points are so that I, as an instructor, can say, “Alright, if you find that you like study groups because you really need somebody to help explain things to you, that is a sign that we need to help you understand the material more effectively, and get you extra help.” So both research and being a good practitioner as Sheela says.

Rebecca: I think one of the things that we don’t always recognize when we’re introducing students to strategies that are really new to them is that they’re learning that in addition to the course content at the same time, so there’s an extra layer of things that need to be learned and we need to do spaced practice and things on that, too. [LAUGHTER] Start building some habits and remember that that’s a thing that they’re also learning and remind them that that’s also a thing that they’re learning and that accounts for some time. It takes time to adopt a new set of practices. It takes time to plan [LAUGHTER] or whatever.

Adrienne: And I like to give students a small amount of points are doing it, because it’s part of the work of learning and I want to reward that, and frankly, they need the points.

Sheela: sAnd some study trategies are also discipline based. So it’s very difficult for our students, especially when they’re freshmen and sophomore, and they are in three or four different types of classes. Some are STEM-based, some are non-STEM-based, maybe their non-STEM faculties guiding them to read and create some kind of a graph or some kind of a writing narrative. And here in our STEM class, I’m saying, don’t just spend your time reading and rereading, while the other faculty is sort of giving them a different point of view.

Adrienne: They have a lot of bosses.

Sheela: Yes. And that can be very difficult for them, because they’re like, “What do I do? My other teachers telling me this, and this teacher is telling me this, and I don’t know, and I’ll just do what I know to do, which is flashcarding,”

John: We always end with the question: “What’s next?”

Sheela: To me, what I would like to do is continue this kind of education research, which is classroom based, that also has an efficacy for student improvement. These three are very important to me. But I also have an overall goal because of the CC Bio Insights network that started me to think about biology education research, is to make sure that these kinds of questions that are community college centric, asking questions that are based from a community college classroom, are also being part of the education research and part of the biology education journals published so that people can actually see what’s going on in these classrooms, and perhaps build some credibility to the work that’s done in this area. So that would be sort of my big picture, giving back to the overall community of community college and how it affects higher ed.

Adrienne: That’s great, Sheela. Yeah, what’s next is always iterations on improving my teaching, new projects for research. But I thought working with Sheela on a project like this was super helpful, both for my career, because it benefits me to publish… that’s a really important part of my job… and I have a community of people with statistical skills and experience publishing, that if I can bring that to a partnership with a community college faculty person who has access to community college students, that’s a important connection that the community college students can benefit from the research finds for them, the faculty member in the community college isn’t overwhelmed attempting to learn a whole bunch of additional publishing skills that they don’t need for their career advancement, but they still want the message to get out that I can carry some of that burden in a way that benefits me. So it’s a win-win for me, for Sheela

Rebecca: Well, thank you so much for joining us. And we look forward to hearing what else you learn.

Adrienne: Thank you so much for having us.

John: And this is really important work that you’re doing because we lose so many students from the STEM fields, and the students we’re losing are the students who could gain the most if they were to be successful in the STEM fields because the rate of return to a degree in the STEM fields is so much higher than it is everywhere else and we face some serious shortages in these areas. So it’s an area in which the research could be really beneficial to a lot of people.

Adrienne:: We’ll do what we can.

Sheela: Yes, keep marching along and carry more with us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

186. Super Courses

Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, Ken Bain joins us to explore examples of courses that do this well.

Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021

Shownotes

  • Bain, K. (2004). What the Best College Teachers Do. Harvard University Press.
  • Bain, K. (2012). What the Best College Students Do. Harvard University Press.
  • Bain, K. (2021). Super Courses: The Future of Teaching and Learning. Princeton University Press.
  • Andrew David Kaugman, Books Behind Bars
  • Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of personality and social psychology, 22(1), 113.
  • Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of educational research, 71(1), 1-27.
  • Perusall
  • Hypothes.is

Transcript

John: Students often see our classes as boxes that they need to check in order to graduate. By reframing our courses around fascinating big questions that students can connect with, we can help our students recognize the value of these learning experiences. In this episode, we explore examples of courses that do this well.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Ken Bain. Ken is an award winning teacher, the founder of the teaching centers at Northwestern, New York, and Vanderbilt Universities. He is the author of two very influential prior books, What the Best College Teachers Do and What the Best College Students Do. His newest, Super Courses, was released in March 2021. Welcome, Ken.

Ken: Thank you very much. Glad to be here.

John: We’re really glad to talk to you. You visited Oswego a few years back and people are still talking about your visit.

Ken: Oh, wonderful. I had a wonderful visit.

John: Our teas today are… Are you drinking tea?

Ken: No, my doctor won’t let me do that, and I haven’t had a good cup of tea in… oh my goodness… many, many years.

Rebecca: Oh, that would make me so sad.

Ken: Yes, indeed, it does. Me too. I can’t drink tea… anything that has caffeine in it.

Rebecca: Ah, total bummer.

Ken: Yes, it is. [LAUGHTER]

Rebecca: I have about the strongest caffeinated Irish breakfast tea you can have. [LAUGHTER]

Ken: Oh, my goodness. Well, the last cup of coffee that I had was in 2002. I remember the date. That’s because it….

Rebecca: Oh, no…

Ken: …a traumatic experience, to go cold turkey.

John: Actually, that’s how I started drinking more tea. I had to cut out caffeine, so I started drinking herbal tea.

Ken: Well, I do drink herbal tea from time to time. I just don’t happen to have a cup right now.

John: I am drinking Tea Forte black currant tea today.

Ken: Oh, wonderful.

John: It’s really good.

Rebecca: So we’ve invited you here today to discuss Super Courses. Could you tell us a little bit about the origin of this book project?

Ken: Well, my wife and longtime collaborator, Marsha Marshall Bain, suggested that we do a course around the Invitational syllabus, as we’ve come to call it, what we used to call the promising syllabus. And we began collecting those syllabi from around the world and began looking at them. And in the midst of that endeavor. Peter Dougherty, who is the longtime director of the Princeton University Press, contacted us one day and said, “Would you come down to Princeton, and I’ll buy you lunch?” So that sounded like a great invitation. And we went down. And he asked us what we were working on. And I told him and he said, “Oh, you’re looking for super courses.” And that triggered a whole avalanche Of reconsiderations of what we were doing, and we shifted the Invitational project over to the super course project, and began looking for courses that offered what we had been calling a natural critical learning environment. And we began that project back in, I guess, late 2007.

John: Maybe before we talk about your new book, you can talk just a little bit about the concept of an Invitational syllabus, since that was the origin of this project.

Ken: Oh, sure. It’s the idea of inviting your students into the class, rather than requiring them to come. And rather than focusing upon topics, it focuses upon big and enticing questions, so that the Invitational syllabus begins with an intriguing question, an important question, a beautiful question that students find so enticing, that they say, “I want to be a part of that.” And it becomes a self-motivating experience. So that’s part of what we meant by a natural critical learning environment, is the creation of that self-motivating experience where students would pursue things, not because someone was threatening them with a bad grade, or because they were just looking for credit, but because they became deeply interested in the question.

Rebecca: Can you expand a little bit upon the idea of the natural critical learning environment beyond just the Invitational syllabus?

Ken: Sure, we now have identified, oh, I guess, about 20 some odd elements of what we call a natural critical learning environment. And the first and most foundational of those elements is that it’s organized around those intriguing questions. And its intention is to foster what the literature calls deep learning, that is learning in which students think about implications and applications of what they’re learning and the possibilities of what they’re learning. It’s learning where students look behind the words on a page and think about all of those implications and applications and possibilities and how things are connected to each other. So that’s the foundational element and the chief goal of the natural critical learning environment is to create an atmosphere where students can, and will likely, pursue that deep approach to learning and they develop what we call deep intentions to learn. But, how do they do that? How do we get them to that point? So, what is the natural critical learning environment? Well, it’s an environment where they can try, they can come up short, and get feedback, and try again, without penalty, without any kind of situation where they are punished for coming up short. In other words, if you think about it, it’s the kind of learning environment that we expect, as scientists and as scholars. We try out things, and if they don’t work, if the data doesn’t confirm our hypothesis, we modify it and try again. And we’d be terribly insulted if our first effort out of the box was… and people would say, “That’s nonsense. That’s crazy. Go away.” We try things, get feedback, and try again, and so that’s what the natural critical learning environment does. It’s also an environment where students can work with each other. People learn in community arrangements, where they work with each other to grapple with the problems. And they learn… and this is another key element of the natural critical learning environment… they learn by doing. Sometimes that means learn by teaching. And by teaching, we don’t mean necessarily that they stand in front of a mirror and deliver lectures. In fact, the teaching that they develop often doesn’t even include lectures, it includes a way of fostering very deep learning on the part of other people by creating dialogues, creating exchanges around big questions that move students toward a deeper understanding and a deeper application.

John: Going back to that question of the big questions, because that’s an important part of the approach. I think you talk about that both at the level of the course as a whole in the Invitational syllabus, but also when you’re devising individual components of your course. Could you elaborate a little bit on what faculty should think about when trying to select those questions?

Ken: Yeah. And it’s more than selecting them. It’s framing them, and framing them in a way that will intrigue students. Now, some of the best super courses we came across were questions that sometimes began with questions that were much larger than the course and much larger than the discipline. But in the course of students pursuing those big questions, they discover that “Well, I need to learn chemistry to answer this question,” or “I need to learn history,” or maybe “I need to learn both,” because many of the super courses were multi-disciplinary, built around a big and complex and interesting fascinating question. And then the students would devise ways of trying to answer that question. And the professor would build an environment where they could progressively tackle those questions. They can run from the very simple to the very complex, one of my favorites, and one that I’ve talked about so much, and actually written about, going all the way back 10, 15 years ago, is one that we do mention, briefly, in this new book, but it’s joined by other really exciting examples. First, that old example, it comes from 2006. And there was a professor at Princeton at the time, he was a political historian and political scientist, and who wanted the students to examine the impact of that period we call reconstruction, in period from roughly 1865 to 1877, and to ask themselves, what kind of impact did that period have on subsequent political and social developments and political institutions? Now, as a historian, that’s a very intriguing question to me. [LAUGHTER] But I don’t think you will find many undergraduates who are just dying to pursue that question. So she didn’t ask that question initially. Instead, she built a course around a question that she knew was already on the minds of her students. Now you think about what was on the minds of students in the fall of 20006? …A big question. When I ask American teachers these days, they often can’t remember. [LAUGHTER] I recently did a workshop in China, and in far southwestern China, and they got it immediately. They remembered. But, the question was, basically, “What in the world happened with that disaster we call Katrina?” Now, there’s a lot of evidence that that question became the dominant question in American politics in 2006, 2008, and helped determine the outcome of the election in 2008. You look at Mr. Bush’s numbers of approval, they fell off the cliff after Katrina. So what caused that disaster? So she organized a course, she called “Disaster: Katrina and American Politics.” And students signed up immediately. It became an extraordinarily popular course. Well, how do you get from there to an examination of political history? Well, it happened on the first day. She went into the class, and the first question she asked her students was, “When did the disaster begin? Did it begin in August of 2005, when the storm surge hit New Orleans? Or did it begin in 1866 with the beginning of reconstruction in the Crescent City?” And with that question, she transformed their interest into her interest, and it became the driving push of the whole course. But let me give you another broader example of the book, A guy by the name of Andrew David Kaufman, who teaches at the University of Virginia, about a dozen years ago, organized a program he calls Books Behind Bars. His field is Russian literature, late 19th and early 20th century Russian literature. And that literature is quite famous for asking big questions, questions about: “What’s my purpose in life? What’s my destiny in life?” So what he does in this course, is the help students go into a maximum security correctional facility for young people, people the same age as the UVA students in the course. When they go into that prison, and they help those other young people confront those questions, by reading Tolstoy, by reading Turgenev, raising the questions, and then struggling with them in a class that they do for them once a week, ensure they learn Russian literature, by teaching Russian literature, and not by lecture, but by creating an environment, a natural critical learning environment, where their students, the residents in the correctional facility, will learn just as deeply as they will. And it’s a transformative experience, for both sides, and it changes lives, and it’s self motivating. That makes sense?

Rebecca: Yeah, these are really powerful examples. And I love that both of them have really strong ledes with the course title.

Ken: Yeah. And if you’ll notice also that both have appeal to a sense of altruism. And we discovered that many of these super courses do just that, even in fields like physics and engineering. They do things to help other people. One of our favorites is a course that some high school girls in a high school in northwest Los Angeles, developed for themselves. And the only help they had was they were invited into this program and invited to come up with a project. And they live in a relatively poor neighborhood. They said, “Well, the biggest problem in our neighborhood is homelessness. And we see the homeless out on the street and in the park and under interstate 5 that runs near the high school. And what we want to do is we want to create a portable tent that is solar powered, so they will have heat and the cooking facilities and light and so on and so forth in their tent.” Now to do that, they had to learn engineering. So they organized their own courses, they organized their own sequence of topics that they would pursue. Now they have some guidance. The teachers over there kind of giving them hints or answering questions: “Should we pursue this next?” But they learned everything from electrical engineering to programming, and lots of things in between. But they also learned just the basics of being an engineer. That’s transformative. They created it, and therefore they took ownership of it. Now, the super courses, and the super institutions that we studied, immerse a lot of what they do in the research on human motivation, research pioneered by Edward Deci and Richard Ryan. And they argue, in their well documented research papers, that human beings have three basic needs and that if you meet those psychological needs three basic psychological needs… we have physical needs that go beyond these… the three basic psychological needs. And if you meet those psychological needs, people are just naturally motivated to try to learn. You don’t have to stimulate it, it just occurs naturally. And the problem is often that the way we set up schooling for people doesn’t meet those needs, it actually counters those needs. And so we get classes full of uninterested students, students who are signing off and not really becoming involved. And to address that situation, many of these courses deliberately use Deci and Ryan’s work to build an environment, where, what shall I say, where people are just naturally driven to do what they need to do. Those three needs, by the way, are: a need for autonomy, that is, we like to be in charge of our own lives. We don’t like teachers being in control of our learning. We want teachers to help us with it, that’s different, but not to control that. And beyond autonomy, there’s also a sense of competence. So people, if they feel like if they don’t know something, they can learn it. And they feel that what Carol Dweck calls a growth mindset, that mindset that says, “I may not know this, yet, but I can learn it. I may not know calculus, yet, but I can learn it.” While the person with the fixed mindset says, “I’m not a writing person,” “I’m not a computer person,” “I’m not a whatever person,” and they give up. And they don’t try to learn and to push the envelope. And then finally, it’s the sense of relatedness, people like to be part of a broader effort, and an effort in the super courses that’s often larger than the classes itself, larger than the discipline… that they take on these large projects, because they believe that it can make a difference for themselves, and for other people whose lives they will affect.

John: One of the things that’s challenging, though, for a lot of faculty, is that we do have to assign grades for all of this. So we know ultimately, students are going to get these grades. And that tends to lead to more of a reliance on extrinsic motivation. What can faculty do to provide that sort of encouragement and to help create a growth mindset when students are going to struggle with some of the material at first?

Ken: Yeah, it’s to give them lots of opportunities to try, fail, receive feedback, and try again. Now, that seems really daunting to many faculty members. They say, “Now look, I have to two hundred students, I can’t do that, for all of the students.” But there are ways of doing that. And that’s one of the things that we explore in the book. It is difficult to describe, so I won’t attempt to do so in the conversations here. But, the courses develop ways for students to give feedback to each other. Sometimes they have students make an argument about their own learning, and then have each other to assess that argument and make an argument. And that second part that I mentioned, is really an important part of the natural critical learning environment, that it’s an environment where people deliberately learn to give themselves and each other feedback. So they set up the whole system of marks around that idea that students were going to give each other feedback on how well they’re doing. And they’re going to give themselves feedback. And that they learn to assess their own efforts and work through those. Often, in the course of the term, credit is often given for participating. That is, if you do the work, you get credit for. And only at the end, do you approach anything like a summative judgment that we usually call a final grade. One of the things that we do in the book is to explore the history of grading. And we do that to help people see that grading is, for one, a fairly recent invention in education. The idea of putting a number or a letter on someone else’s thinking, that didn’t emerge until fairly late and really didn’t become entrenched until the late 19th to the early 20th century, and that changed everything. So I want people to see this in that kind of context, that there’s nothing natural or automatic about having the traditional approach to grading. And so what people have done in the super courses, is find ways of saying “Okay, now you’ve joined a community, you’re going to be helping each other to learn and you have responsibilities toward that community and to help each other to assess each other, to give each other feedback… substantive feedback, not scores, substantive feedback to one another. And we’ll try to give you feedback as well, maybe as a group, maybe individually in smaller classes, but to give you that opportunity of trying, coming up short, and being able to try again, without that affecting your overall final grade. And then the final grade is based upon an accumulation of lots of things, and perhaps a final project, a final paper, a final presentation, or something of that sort, rather than just simply accumulating, you get 10% on this and 15% on this and 40% on this aspect of the grading.

John: How can we help students embrace the concept though, of productive failure, that process of trying something, making mistakes, and then learning from that experience. Because that’s something that many of our students don’t naturally come to, because many of them haven’t seen it before up to the point when we have them in class.

Ken: Yeah, exactly. And I think one of the keys is to provide them with very dramatic and enchanting learning environments, at the very beginning of their experience, so that the students say to themselves, “This is going to be different.” Let me give you an example of what I’m talking about… my favorite example of this category. It came from a program that was offered, again, in a secondary school. It is called city term. And it brought students from around the world together right outside of New York City, and they use New York City as their classroom. And the first Saturday in the program, they’re invited to go on a scavenger hunt. And they’re given a list of items that they might look for in New York City. One of them, for example, that they often use was: find the first wooden escalator. So students go off in groups, and there’s a teacher that goes with them, but the teacher doesn’t interfere, and just keep them safe. And beyond that the students go wherever they want to. It’s a wild and exciting adventure. And then at the end of the day, they end up in Central Park on a picnic, and they discuss with each other. “How did you find that escalator at Macy’s? What questions did you begin to ask yourself? Who did you talk to? How did you reason through the process?” And by sharing ideas with each other, what they’re actually doing is learning good research techniques. That’s a wild way of learning good research techniques, to say the least. But it’s something that the students will always remember. And they will latch on to that. And they will latch on to the course now, because that first experience was something that was quite dramatic to them. Now, we don’t all have the opportunity to use New York City as our laboratory or our classroom, and to take students on a scavenger hunt. But we can imagine creating a first assignment, and I’m reluctant to use that word “assignment,” because we found often that these courses don’t talk about assignments, they talk about opportunities, and invitations to students. It’s so exciting that it begins to break down all of their sort of stereotypes in their mind about what’s going to happen in a class. So in Andrew Kaufman’s Books Behind Bars course, they’re first asked to apply for the course. So they have to explain why they want to be in the course, and then helps to begin to break down barriers. And then the first day in the class, he begins to break down the barriers by first telling them about a three minute story about a young man who read a little short story by a guy by the name of Tolstoy, Leo Tolstoy, and as we called him in the West, and about halfway through the story… in other words, about a minute and a half end of the story, the students began to realize that it’s a story about Gandhi. And it’s a story about an important transformation in Gandhi’s life as a result of reading a piece of literature. And so Professor Kaufman says to his student, “We want you to think about a point in your life, when a piece of art, maybe a piece of literature, maybe a painting, maybe a song, but some piece of art had a deep impact upon you and your thinking.” And the students began to discuss with each other. And they began to realize first, that this isn’t going to be a course where the teacher just talks to them and they take notes and then later take a test on whether or not they can recall the notes that they took. But it’s going to be a class that they will dominate, that they will do most of the talking and most of the thinking, and by creating a different kind of environment, you then can move to ultimately getting them to think about such questions as how are you going to assess yourself? How do you know whether or not you’re making progress, and whether or not you’re learning and you’re learning deeply. And the key point here is helping students to learn what it means to learn deeply, that learning deeply is not the traditional strategic learning… “oh, I learn this for the test. I’ll make an A on the test and I’ll make an A in the class.” No. it’s self-driven learning, where you begin to look behind the scenes, where you intend to look for ways in which this course can transform you and transform your thinking.

John: So essentially, I think what you’re saying is we need to help encourage students to develop more reflection on their work and on their learning process.

Ken: Yeah, I think so. I think that’s the key is getting students to think about their own learning, and help them with categories that will enable them to think deeply about their own learning… categories like deep learning, versus strategic learning, or surface learning. The strategic learner just wants to make straight As [LAUGHTER] and they’ll do whatever is necessary to make those straight As. The surface learner just wants to learn enough to pass the course, to be able to perform on an exam or write a paper or whatever it is that’s required of them. But neither one of those two leads to deep intentions, that I deeply want to understand how calculus works, and how it can help me in understanding the world in which I live, and how this applies to me, and my field and how it applies to my major, even though I’m not a mathematician, and how I can change the way in which I think. So developing those deep intentions. and fostering that deep development of intentions, becomes extremely important.

Rebecca: One of the things that you were just mentioning in terms of the strategic learners and the surface learners is how much many of our courses are probably structured with them in mind, rather than a deep learner in mind, and that we perpetuate these kinds of learners rather than deep learners based on our class structures.

Ken: Yeah, exactly. Exactly. And I think we began to break away from that, by the way in which we frame the questions we raise. You think about the example of Kaufman’s course. Or let’s take an example from physics. Eric Mazur, at Harvard, has pioneered a lot of the elements of the natural critical learning environment and super course. And the students in his course do not learn physics, by listening to lecture, boring or otherwise. They learn physics by doing physics. They do three big projects. And each semester of the course… it’s a two-semester course, some students take only one semester… but in either semester, they do three big projects and they’re massive projects. And they work with a team. Each student’s is in a team of about five or six students. And they work together to try to attack a problem. And each of the projects has a back story. For example, you’ve been contacted by a charity that was created in Venezuela, by a well known philanthropist and musician, who became quite convinced that music, and symphonic music in particular (being part of a symphony orchestra), is a transformative experience that can help very poor people rise out of their poverty, and to develop a different mindset that enables them to conquer some of the economic circumstances they face. It’s a program that now has about a million students worldwide who are engaged in it. But it has a problem, namely that some of the students are so poor, that they cannot afford to buy real instruments. Now, you’ve been studying waves and music is made up of waves. So, your team has been invited to create new kinds of instruments that can be made from things that you find in the junkyard. Now that new instrument has to be able to be tuned over to different octaves, has to stay in tune for a specific amount of time, but by creating these new kinds of instruments, you can help the young children. Now, that’s a compelling project. They learn physics and the physics of waves actually doing the project and demonstrating that they can do it. Another project is more fun than anything else. Do you remember the old Rube Goldberg cartoons?

John: I do. I don’t know if Rebecca does. [LAUGHTER]

Rebecca: I know what they are. [LAUGHTER]

Ken: Ok. There was this guy. He was an engineer and he was also a cartoonist from San Francisco. He used to draw these cartoons, where you would attempt to do something very simple, like crack an egg, by a very elaborate piece of machinery, where balls would roll down ramps and would trigger gates that would open up other gates that would cause other balls to fall, and so on, and so forth. So it’s a very elaborate, unnecessary project. So on the Rube Goldberg project, so students are invited to create a Rube Goldberg type device to crack an egg… [LAUGHTER] something you can do very simply by knocking the egg against the edge of your kitchen counter, or your kitchen table. But no, you had to create this very elaborate project. Well, to do that, all of Rube Goldberg drawings and inventions were based on physics. That ball rolling down a ramp at this angle will acquire this speed, and it would open this door and would result in this hammer hitting this hammer that would cause this ball to roll down this ramp… and that would, etc, etc, etc. So you had to know the physics in order to create these absurd projects. But there’s a lot of fun in that. A lot of fun. And the first project they do, I think, is one was just strictly on fun. They build a racing car, and then race each other, like many of them did when they were back in the fifth grade. And they have great fun in doing that. Another one is one where they have to design a lock that other people can’t open. And they get points for 1. being able to keep other people out of their safe, and for being able to crack the lock of other peoples’ safes. So a lot of wild times. Now there’s a textbook that stands behind all of this. And the textbook is written by the professor. And it lays out everything that he might otherwise have said to them in lecture, and more. Now, how do you get students to read that textbook, 1. by creating these enticing projects, but another way you do it is by making reading of the textbook into a social experience. So Mazur and his colleagues created a program which is now available to everyone, called Perusall. And, using Perusall, students can read material together. So in the sense that, “Okay, we’ve got an assignment, we need to read to chapter two. We’re all reading it together, you’re reading it on your computer, I’m reading it on my computer. And as I read along, I may have questions. So I’ll highlight that text, I’ll raise the question.” It’s like writing in the margin, but everybody can see what you wrote. He organizes the groups into groups of 15 or 20 students apiece, and they can read each other’s comments and they can respond to each other’s comments. And to participate in the class, to be a participant in the class, what it means is that they will keep up with offering the comments to each other, and their comments on the text itself, making comments on what’s written there, to raise questions. to answer a question, so on and so forth. And what they found is that reading completion goes from 40 something percent at best, all the way up to 95 to 100% completion.

John: I’ve been using Hypothesis in my classes for the last three years, and nd it’s a very similar type too…

Ken: Yeah, there are several others out there.

John: And students really enjoy that, too. They enjoy seeing what other people are raising questions about. They enjoy answering questions for each other and posing questions to each other, and It seems to make the reading process much more engageing when it becomes this social activity.

Ken: Exactly. Well, that’s the whole idea. And then of course, behind that is this set of really intriguing, interesting, fascinating projects. And there’s a course at an engineering school in Massachussetts that offer a course, for a long time. They no longer offer the course unfortunately, but they offered it for over a decade, I think it’s called the “history of stuff.” [LAUGHTER] And the first day of class, the students go in and they see on the front table stuff. It’s the kind of stuff you might find by going down the aisles of Walmart [LAUGHTER] and picking things off the shelf, just a wild assortment of things. And they’re invited to come down and pick out one of those items. And then to begin to explore it, explore its history. Why was it created? What was it created out of? What materials? What kind of implications does its creation have for society? Does it just clutter up society and create a backlog of unrecyclable material that creates environmental problems of one type or another? Or exactly what is it? Students had a great deal of fun just exploring stuff. And the course ended up by looking at some of the history of technology through the lens of a well known American patriot, Paul Revere. But Paul Revere was also an expert in metals. And so they explore engineering of metals through the eyes of Paul Revere. And it becomes a way of mixing disciplines in a way that makes each discipline more intriguing and more interesting. Rather than “Oh, you study this, then you study that, you make no connection between the two.” Say, one more example?

John: Oh, sure.

Ken: There was a course we looked at in southwest China, and we went to the school, Southwest Jiaotong University, in Chengdu, and it was a course organized by a young woman who teaches physical education. And first day of class, the students are invited to think about what kinds of sports they enjoy doing. is in rock climbing. Is it soccer? Is it basketball? What is it? Now, can you imagine creating an exercise device that will make you a better soccer player, or rock climber, or whatever it is that you want to do… your favorite sport? And then the whole class goes to a sports equipment store, and they began to look at the equipment that’s already there. And then began to think about, “Okay, how can I create something better?” Now, as part of the team, it’s not just this PE teacher, but it’s also other people. Ah, you need someone from biology perhaps, to help them think about… “Well, what kind of exercising do the human muscles need? What kind of social environment do you need to create here?” So, you need other experts. And if you want to make this a product that can go on the market, maybe you need a marketing professor, who can help you devise a marketing plan of the new product that you’re creating. I told this story to my broker several years ago, and his response to me, was: “In Communist China?” And I said, “Yes, they have a market economy just like we do. And they’re interested in marketing and learning marketing. And they have marketing professors, just like we do.” And so they creates this environment where students learn by doing. And they learn by mixing disciplines, rather than keeping them apart. Much more interesting.

Rebecca: I love the move towards more interdisciplinary work….

Ken: Yeah.

Rebecca: …something that I feel really connected to, but it really gets people I think, more excited about different disciplines when they’re more intertwined, because we understand how they’re related to one another.

Ken: Yeah, exactly. So if you’re going to study the human brain, for example, how do you make it interdisciplinary? One of the professors, we studied taught in the medical school and taught medical students about the brain. But she was asked to create a course for undergraduates. This was at Vanderbilt. And so the course on the brain for undergraduates mixed every discipline you can imagine, together, because, as she argued, everything is connected to the brain. So you might be studying music, if that’s your interest, or whatever your interest might be. You might be studying ethics, if that’s your interest, and then you’re encouraged to think about what part of the brain handles ethical questions? What part of the brain helps you to appreciate and understand music? What part of the brain helps you to do this or to do that? So they’re studying all the aspects of the brain, but they’re also studying all these other disciplines, from Holocaust studies to a wide variety of other things, and raising deep ethical questions along the way. And she offered this course for 10 years, and it was a transformative experience for most of her students, the overwhelming majority of them, breaking down stereotypes and prejudices and helping them to also think more deeply about how their brain operates.

Rebecca: Sounds like there’s a lot of classes I should sign up for.

Ken: Exactly. I thought at one point, I was talking to some high school students about where they want to go to school. And I said, “Well, it’d be wonderful if you could go to a school that would mix all these super courses together. Because they’re strung out all over the world.” Maybe there’s a way of doing that virtually I don’t know.

John: Or maybe, as a result of your book, and other similar work, perhaps more faculty will start doing this type of thing and more of their courses.

Ken: Yeah, and perhaps, in designing a curriculum that includes professors from a wide variety of different disciplines, and students from each of those disciplines, working together in small groups, to tackle problems of physics, and then later tackle problems of the brain or tackle problems of history, or tackle problems of well, you name it… and have a opportunity to sort of tour super courses around the world. That would be a wild experience.

John: I still remember examples that you used here when you spoke at Oswego. And I remember examples when I first read your first book on what the best college teachers do, a while back, in large part because you weave in narratives, along with the theory and the reasoning behind these concepts. And I think the use of narrative helps makes the story much more interesting and helps raise curiosity and makes things much more memorable. Is that something faculty should strive to do in their own classes?

Ken: Yes, I think so. And the professor, I was just mentioning at Vanderbilt, I think, did that and created a course that was part history and part neuroanatomy and part philosophy and part literature and part music, but they’re all around narratives of one kind or another. Yeah, I think so. I think creating that narrative. Human beings love stories. And if we began to understand things, in terms of stories, then it becomes much more memorable to us. And we remember what we learn. And if you think about learning, it contains at least these three major aspects: we’ve got to encounter new ideas and procedures, and so forth, but we’ve got to encounter new material, there’s the encountering part. And the second is the making sense of it part where we relate it to other things that we’ve learned. And then the third aspect of it is retaining it long term. So we remember what we remembered, what we learned. And I think encountering all of this in stories, makes it much more memorable. But I think what the super courses do is they have students read stories to learn physics or history or other kinds of things. But they don’t tell them those stories orally, for the most part, they do not use lecture, to do that first aspect, that is of introducing the material. Usually, that’s all that happens in the classes, you’re introduced to the material and lecture, and you never get around, you never have time in class, for those second aspects of the “making sense”part of and the things that you might do to retain it. So super courses are built in a way that they spend their time working on those other aspects. Because the first one, the one of conveying the new information and ideas to the students, that can be done with reading, with films with other ways. But the part of struggling with meaning, with the teacher and with each other, that’s much more complex, and that requires a different kind of approach. And that’s what the super courses offer.

John: This project began with a collection of syllabi, and we should probably note that those syllabi do make it into the book as an appendix. So, not only do you have the stories of how these classes work, but it provides faculty with examples of how these things are implemented. In an Invitational syllabus.

Ken: We took excerpts from some of the syllabi, not all of them. But from a few, to give people illustrations of what we’re doing: one from math, one from the sciences, and one from the humanities.

Rebecca: We always end by asking, what’s next?

Ken: [LAUGHTER] Good question. Well, I have on my agenda, and I’ve been working on, a book aimed at parents. And the working title of the book sort of summarizes the whole idea of the book. Although the working title is 11 words long, andt hat is way too long. But we’ve got to find a way to achieve the same thing with a shorter title, but it’s: How to Help Your Kids Get the Best Out of School. Now we chose those words carefully, because the first task is defining what we mean by the best. And, in part, it means learning to learn deeply. So how do you help your kids to do that? And we chose one of many words we might have used for kids. We said “kids,” ‘cause it’ss short and to the point. We’re trying to shorten bold type as much as possible. And I’m working on that with a colleague, Mindy Maris, and we have a due date with Harvard press of 2022. So, that’s coming up rapidly. So we’ve got a lot of work to do over the next year and a half. But we’ve already done quite a bit of work in organizing that, and so forth. So that’s the next major project.

Rebecca: That sounds like an exciting addition to the collection that you’ve already have out, and rounds out the offerings.

Ken: And then somewhere in the far distant future I play out entirely, I would love to take all the we’ve learned and how to understand the best in any field. And that was a process in itself. How do you define the best and how do you collect evidence that something is better. I’d love to do a book that might be entitled: What do the Best Coaches Do? [LAUGHTER] and describe good coaches in a wide variety of different sports. But that would be my swan song, if I ever get around to it.

Rebecca: Thanks so much for joining us today and sharing some insight into your newest book. I know that a lot of our listeners will be looking forward to reading it soon.

Ken: Well, I look forward to hearing feedback from your listeners. And as we said, toward the end of the book, we hope that at some point, every reader will say “I wouldn’t do it that way. I’d do it this way.” But when they say that, we hope that they will base that judgment on strong evidence that that presents, whatever alternatives they come up, presents a better learning environment than the one we describe in the book. But, we hope this idea of a super course, is something that is organic, it continues to grow. And five years from now, somebody will summarize something about super courses today, meaning the super courses, 2027 or 2030. And may describe a much different book than the one that Marsha and I wrote. But it’s an organic process. And we’re looking forward to the conversation w e hope that the book stimulates.

John: Well, thank you. It was great talking to you again, and we’re looking forward to sharing this with our listeners.

Ken: Thank you.

Rebecca: Thank you so much for your time. We really appreciate it.

Ken: Oh, my pleasure. My pleasure. Anytime.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

176. Critical Thinking

“I want my students to think more critically” is a familiar statement in higher education, especially when we mix in conspiracy theories, pseudoscience and fake news. In this episode, Dr. Linda Nilson joins us to discuss practical techniques faculty can use to help students develop the skills necessary to become critical thinkers. Now Director Emeritus, Linda was the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Infusing Critical Thinking Into Your Course: A Concrete, Practical Guide as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

Transcript

John: “I want my students to think more critically” is a familiar statement in higher education, especially in a social media environment filled with conspiracy theories, pseudoscience, and fake news. In this episode, we discuss practical techniques that faculty can use to help students develop the skills necessary to become critical thinkers.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Linda Nielson. Now Director Emeritus, Linda was the Founding Director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Infusing Critical Thinking into your Course: a Concrete, Practical Guide, as well as many other superb books, book chapters and articles on teaching and learning. Welcome back, Linda.

Linda: Thank you very much for having me back, John.

Rebecca: Today’s teas are: …Linda, are you drinking tea?

Linda: I certainly am. And this is Twinings Berry, multiple berries.

Rebecca: Oh, yum.

Linda: Yeah. It’s delish. And I’m so glad you’ve helped to get me back into tea. [LAUGHTER]

Rebecca: As everybody should be,

Linda: Yes, yes, yes. It’s a wonderful thing.

John: I had some of the Twinings mixed berries early today. But I’ve got a chocolate mint black tea, which is really good.

Linda: Ooh, chocolate.

Rebecca: Chocolate anytime of day is excellent.

Linda: Yeah, really.

Rebecca: It’s an important food group. [LAUGHTER] Just like tea. I’m drinking that palm court blend that I have recently started making pots of. It’s pretty decent.

Linda: Great.

John: We just saw a note that you have this forthcoming book on critical thinking. What motivated you to write this book, now that everyone has this universal agreement on facts and reasoning and logic…

Linda: Yeah. Oh, yeah.

John: …and we don’t have a lot of disputes on these things anymore? [LAUGHTER}

Linda: Ha hah… Ah, yes. Actually, politics had nothing to do with it, if you can believe that. Originally, I got interested in critical thinking in the 90s. I decided to give a talk on it for some conference. And I got into the literature. And I said, “Okay, as soon as I get this talk over with, I am never, ever, ever going to do anything having to do with critical thinking, again, because the literature, it’s just a siloed mess.” So anyway, I just put that away. Well, then I guess it was something like eight years ago or so, Clemson University, in its infinite wisdom, decided to select critical thinking for its QEP, its Quality Enhancement Plan. And I thought, “Oh, no, do you know what you’re doing? Do you have any idea how difficult this is going to be?” And of course, I figured, like, I’m going to be doing workshops, so I got to get back into this literature. And I realized what I had to do was synthesize it for the faculty, because why should I inflict a literature’s flaws and warts on them. So that’s what I decided to do. Well, then, as it turned out, Clemson wasn’t the only university picking critical thinking for a QEP, and so I started getting invitations to do workshops, so I got on the road, and that’s always fun to do. I was back into it, but trying to make it as practical and as easy to implement for faculty as possible. So that was my goal, is always to make the faculty member’s job easier. So, I was comfortable with that. Well, a couple of years ago, David Brightman from Stylus contacted me, asking me to write a book on critical thinking. And I thought, “Well, I’m kind of enjoying being semi-lazy and just traveling around, oh, what fun.” And then I thought, “Well, look, I don’t know about this. But let me just start outlining the book and see how it feels.” Well, it felt really good. I was amazed. [LAUGHTER] And I thought, “Okay, lazy is not so interesting to me anymore. I’m going to write this book.” Well, he was happy. I was happy. And so I immediately started writing it, and because I had these workshops to work off of… updated by reading of the literature, of course… but other than that, it was probably the easiest book I’ve ever written. And so, this is fun, this is neat, this is great. And so I finished it really very quickly. And I was glad to do it. I finished it in four months or so… four to five months, the whole….

Rebecca: Wow..

Linda: Yeah, I know. Well, I was a house afire and I had nothing better to do. [LAUGHTER] So anyway, that’s how I was inspired to do it. It was really David Brightman ‘s idea. And if he was excited about it, that kind of got me excited about it. And I knew the need was out there. I knew the need because there were still universities crazy enough to adapt critical thinking as a QEP. I had also acquired some friends in the critical thinking community by then and so “Okay, this is fine. This is great. I’m going to do this.” And so I did it. And so now it’s due out March, maybe April, because you never know about these things. So it’s really happened. [LAUGHTER]

Rebecca: I found that a lot of faculty really believe in the idea of critical thinking, but have no idea how to define it or explain it to students.

Linda: Yes. And that’s a perfectly reasonable reaction for them to have, because of the massive literature out there. And it’s just the idea of critical thinking, it’s such an abstract, squishy, ephemeral idea. And we’re like, “What is it?” …and of course all faculty think they’re teaching it, right? But they’re not ,unless you make critical thinking, an outcome, a specific outcome that makes sense in your course. And you tell your students about it, and you call it what it is, that you’re not teaching critical thinking. It’s not going to happen by osmosis. It’s not going to happen by happy accident. Critical Thinking just doesn’t do that, it has to be a very, very conscious effort. And there’s a lot of literature to back this up.

Rebecca: So what is it? [LAUGHTER]

Linda: Oh, that’s a good follow up question. Well, I’m not going to go through the litany of definitions that you can find out there in the literature. But what I was able to glean from that literature is this: it’s interpretation, analysis, and/or evaluation, for the purpose of making some sort of decision or solving some sort of problem. And that’s it. But you could get definitions out there that go on for a couple of paragraphs, you can get definitions that seem to change color from one chapter to the next. I mean, they’re all there. And I just found it very confusing myself. But again, it was all about synthesis to me… all about synthesis, taking the best from these different silos, these different frameworks, and try to put them together into something that’s easiest, into a pocket definition: interpretation, analysis, evaluation. Now, you can put that in your pocket. And so can students.

Rebecca: I’m sold. [LAUGHTER]

Linda: Alright. That’s wonderful. That’s wonderful.

John: I remember when we introduced a critical thinking Gen Ed requirement here at Oswego, that everyone agreed that that was something that was desperately needed. It’s just everybody had a different idea of what it was. [LAUGHTER] And so eventually, it became infused within the disciplines. But I still think that that discussion of what exactly it is was never really resolved. It was kind of a way of sublimating that whole discussion and debate. So is critical thinking the same across disciplines? Or is it really going to be different depending on your disciplinary context or lens?

Linda: Okay, well, when you’re talking about interpretation, analysis, evaluation, yeah, that goes across the disciplines. But that’s not of much help to faculty. And so I put it in the disciplines, or I should say, in disciplinary clusters, because let’s face it, chemistry shares a lot with biology, right? …just in terms of general approach to observation, its approach to testing, just the general scientific thinking. And the humanities, there are a lot in common. And by the way, sometimes the social sciences pretend to be humanities, when you’re talking about the theories and things like that. Now, the social sciences, I look at them as sciences. So I put them in that scientific cluster. And then they’re the arts. Well, they’re off somewhere else entirely. They’ve got all….

Rebecca: Hey! [LAUGHTER]

Linda: Ah… oh, I’m sorry. You’re in it. But the arts are in a beautiful place. And they’re far more beautiful than the other disciplines. And then there are the, what I call the Applied Sciences, they’re different still. Oh yes, they share some overlap with the sciences. But so what I do is, there’s one chapter where I talk all about critical thinking outcomes by these disciplinary clusters. And now I have a list of different outcomes for each cluster. So take your pick, add more, but it’ll sound familiar to faculty. It’s using their vocabulary, it’s addressing their concerns, the sort of things that they strive to teach, the sort of things that they want their students to be able to do. And so I like to put it in the, at least, disciplinary clusters. Now it’s still you know, yes, you have to bring it down to your particular discipline, and then your particular course. But other than that, there are all kinds of tremendous overlap. But it’s such a different context. Sometimes you’re not even using the same verbs, and you’re certainly not using the same direct objects. I can assure you of that.

Rebecca: Why do faculty think it’s so hard to teach?

Linda: Oh, well, they’ve sure got good reasons. And I have a whole chapter on 10 reasons why teaching critical thinking is so challenging, and it’s got all kinds of reasons. First, there’s the literature, okay, that’s reason zero, okay. And so, I’m sort of, in a sense, telling faculty “Don’t read the literature, just don’t worry about that, or pick one and have fun” … whatever, but that’s not going to help you a lot. And that’s what’s different about my book, that mine’s, again, practical, concrete… here’s what you do… connect the dots. But reason number one is critical thinking about certain kinds of thinking, like that definition I gave you, but also a certain kind of subject matter. And there are courses out there that don’t have that subject matter. There has to be, in this subject matter, it has to be content containing claims, statements that may or may not be complete, valid, or at least the most valid, or the most viable, and for perfectly good reasons as well. For instance, there might be other respectable competing claims, the evidence supporting a claim may be weak or ambiguous, the data may be suspect, the source may have a lack of legitimacy or conflict of interest. So anyway, there are some courses that just seem to just be teaching undisputed facts. Now, if you’re teaching disputed facts, that’s fine, then you could talk about critical thinking. But there has to be some sort of dispute, some sort of competition, something among different claims. And again, there are, unfortunately, some courses that don’t have that… now you can add them in. Another thing about teaching critical thinking… It’s difficult for people to do, for students to do and it’s unnatural for students, for learners, for people to do, because we all want our current beliefs and values and ways of thinking confirmed. That’s what we are usually going for. And then there’s the whole struggle of learning. Learning is struggle, learning is effort. And students don’t seem to accept that. It’s like we’re supposed to make learning so easy. As easy as it was in elementary and high school. Isn’t that our job? …to make learning easy and effortless? No. But in any case, they might fight that: “Well, this is hard. So you’re not a good teacher. And anyway, students also bring biases and misconceptions into the classes that interfere with critical thinking. They’re not aware of these, and so you have to make them aware of them. And students don’t necessarily like that. Critical thinking requires self regulation, or metacognition. If you’re not observing your thinking, there’s no way you can think critically, I’m sorry, you have to question yourself all the time. But you’ve got to be aware of what that thinking is to even talk about it. Here’s another one and it’s nasty. Critical thinking requires the traits of good character. So virtues like integrity and determination and morality and inner strength. And some of our students just don’t have those virtues. Now, there are ways, there really are ways, that you can teach them about good character and bad character. And just the way good character is portrayed in our media and our literature, in movies, television, whatever, it makes good character look good, makes it look more attractive. But students don’t necessarily know what even good character is. And they can’t do self-regulated learning without having some intellectual virtues, like perseverance, like wanting to pursue truth, intellectual integrity, intellectual humility, certain dispositions that are absolutely essential. Otherwise, why bother to do critical thinking? Why put yourself through this? Here’s another thing that critical thinking requires: mental health, emotional health. Now, on the level that you can start increasing students’ mental health is on the level of defense mechanisms. But if you’ve got students out there, and I bet you do, who have personality disorders, they’re narcissists, they;re sociopaths, they’re psychopaths, so they might murder you someday. Forget about them, don’t even have hope for them. They have no interest in pursuing truth at all. That’s why they don’t wind up in the psychiatrist’s office. “What’s wrong?” “Nothing’s wrong,” right? And so what’s wrong with my thinking? Nothing’s wrong with my thinking. So they’re not into this, forget about them. Critical Thinking also requires students to respond to questions. And students don’t always like to respond to questions. They don’t mind telling you what day of the week it is. But beyond that, if they’re going to have to think, they might not be eager to do this. Now, if everybody else is doing it in the class, that helps. It really does. So yeah, there are all kinds of reasons why critical thinking is very difficult to teach, but it’s not impossible, and it’s not mystical.

John: You mentioned self regulation, what are some techniques that we can use in our classes to help students become more self regulated in terms of developing critical thinking skills that let them analyze evidence more effectively and interpret arguments more effectively?

Linda: Yes, well, again, awareness is where you start. So the first thing you want to do, the first habit that you want to develop in your class is whenever you have students give a statement… I don’t mean an undisputed definition, okay, I mean something that’s addressing a claim or making a claim in some way? And you’ve got to just keep asking them: “Well, how did you arrive at that response?” Now, you’re going to sound like a broken record the first couple of weeks, and then students will realize, “Okay, I’m not going to talk unless I have an answer to that question, because I know that Professor so and so is going to ask me this question.” So they start to think about this on their own, and they’re ready to answer it, or they’ll keep their mouth shut. But it’s not so impossible to look into your head as to how you came up with a certain answer. And initially, students will have to say, like, “I don’t know,” and you know what, that’s okay. Because now you have their attention. If they don’t know that that’s a perfectly fine answer, you need to start thinking about that, you need to start observing your thinking. And that’s the first thing you get them to do. Now, in terms of like, with readings, or if you’re having them watch videos and these sorts of things, you can give them reflective assignments to accompany those readings, and for that matter accompany any other kind of assignment. But what you want them to do is you want them to reflect on perhaps their affective reactions to the reading, or what they found particularly important, or what they found particularly surprising when they do a reading and then they have to answer this question or couple of questions at the end. And again, there are no wrong answers. This is like no stress, folks, just I want at least 100 words from you, that students start looking over what they just read and thinking about it from their own viewpoint. What did they find most important? How did they react emotionally? What connections did they make to what they already knew. So it’s making them more aware of what kind of impact that reading had. And those are just the simplest things in the world with substantial assignments, writing assignments, what you want your students to do is sort of a meta assignment at the same time. And again, you can’t go wrong, folks. And all you’ve got to do as a faculty member is just to check these in, just make sure that they did it and if you want 200 words, perhaps, “Okay, describe your reasoning in solving this problem.” or ”How did you reason through this case, in debriefing this case? What questions did you ask yourself along the way? What skills did you improve in the course of doing this assignment?” There’s so many different possibilities. And so, yeah, self awareness, self awareness. And then, of course, after a test when they get a graded exam back, make them aware of how they prepared or how they didn’t prepare. And so it’s making them face the music about, “Okay, what did you expect to get on this exam? And what did you get?” Gee whiz, how do you feel about that? Now, “A” students are gonna “Oh, yeah, I thought I got an A,” or I thought I’d get a “B” but I got an “A,” and so I feel whoopee.” Okay, don’t worry about them. They’re already self regulating, they self regulate to the point of paranoia, they’re the least of your worries. [LAUGHTER] But other students need to become aware of, “Okay, so how did you study? Did you study the way I told you too, which is quizzing yourself? Or do you just reread, reread, reread, reread the way I told you not to, because that’s a waste of time? How many hours did you study? Was this enough? What are you going to do differently on the next exam to do better?” And so students have to think about a strategy, they’re meta-studying in a way. And so they’re thinking about their learning strategies. And they really need to do a lot more of that. Any kinds of like, experiential activities… there are all kinds of questions that they can ask themselves, or you can ask them to write about, “Well, what were your goals originally? And how did they change through the simulation or the game or the role playing?” And this would be true of like service learning or field work or any of these things. “How did you respond to other people involved in the interactions? And why would you respond differently next time?” So there’s a lot of self assessment going on. And this is a good place to start with self assessment. It’s a safe place to start with self assessment, because usually everybody changes their strategy from the beginning through the middle to the end. And that’s good. That’s what they’re supposed to be doing. That’s called learning. [LAUGHTER] But it makes them aware of that learning. That’s how you do it. Yeah.

John: It sounds like there’s quite a bit of overlap between improving students’ critical thinking skills and improving students’ metacognition,

Linda: Absolutely.

John: Is it effectively the same type of strategy used in both cases? Or is it essentially the same process In both cases?

Linda: What’s different about it is with critical thinking, you’re dealing with real disciplinary content. And you’re not necessarily doing that with metacognition. Yes, self awareness is definitely a part of critical thinking. But that’s not all. I mean, you’re actually trying to come to some sort of conclusion or solve some sort of problem, maybe even decide to take some sort of action. You’re examining claims, you’re examining data, you’re examining sources, you’re examining a lot of different things along the way that you’re not necessarily doing in metacognition, but metacognition, it precedes critical thinking. I mean, if you’re not aware of how you’re thinking, how in the world, are you going to come up with a well reasoned approach to a problem? So there’s a lot more. You’re using metacognition, to extend reasoning into something that is disciplinary based.

Rebecca: I’m so glad we’re having this conversation today. Because I’m working on a new class where practicing thinking moving into higher-level classes is the goal of the class. And so I was working on a lot of reflection assignments. So I was just doing a little editing while we were chatting. [LAUGHTER]

Linda: Oh. alright, See, I’m glad this is useful, and that’s the whole thing. I want that book to be useful. And that’s what makes it, I think, different from the other books out there and critical thinking, because most of them are in a particular silo, they’re coming from a particular silo, they’re may be extending that silo in some way. And I was looking for common ground across multiple silos, and then maybe not all of them, but multiple silos, I tell you one thing that they all have in common, they all say that critical thinking involves metacognition or self regulation.

John: And I remember reading a really good book on creating self-regulated learning.

Linda: Oh yeah, that one, yeah.

John: That might be something we’d recommend to faculty.

Linda: Yes.

John: We’ll include that in the show notes as well.

Linda: Okay, good. Yes, indeed, I’m glad I wrote that book beforehand. I really am. [LAUGHTER]

John: And it ties in nicely with a lot of your discussion in the book. In this new book, you’re also addressing how specifications grading could be used. And I know there’s a really nice book on that as well.

Linda: Oh yeah, yeah.I remember that one.

John: You mentioned that when you grade things, that it doesn’t necessarily require a lot of work for the faculty member. Could you just talk a little bit about how specifications grading could be used to evaluate these types of activities?

Linda: Sure, absolutely. First of all, all assignments, not courses, all assignments are graded pass/fail, and for that matter tests, but you don’t pass either a test or an assignment at a C level, you raise your expectations to a B level. But guess what, you’ve got to know what components in that essay or that paper or whatever that design that you want to see to achieve a B level. Now, if you already have rubrics, fine, you can start looking at the top level of the rubric and maybe take a few from the next one down. But those are the things that have to be in the piece of work for it to pass. And by the way, you explain this to students. Now the stakes suddenly become higher for students. So guess what? They read the directions? Isn’t it wonderful? They do, they want to pass. Now, initially, they don’t believe that they could possibly fail at anything because of partial credit. Well, guess what, there is no partial credit here. But they won’t believe it till they fail something. So you always want to give them some get out of jail free cards, so they can maybe fail a couple of times, and then redo the assignment the correct way, because they realize, “Oh, he or she is really serious about this. Oh, goodness. Okay.” So yeah, they really start reading the directions and doing what you asked them to do. And you know what? If they’re worried that they’re not doing it, they visit you, they call you, they email you, they actually ask for clarification, because this means something. And we’ve been lowering the stakes and lowering the stakes and lowering the stakes for years now, because we don’t want to cause them any stress. Well, you know what? We’re going to have to cause some stress. Learning is stressful, sorry. It’s just the way life is. So anyway, when you know what you want to see there, and this is where the thinking is… this thinking it might be before the course starts. So once the course starts, you’re on easy street, because all you have to do is look for those elements in the piece of work that the student hands in. Something missing? You mark which element is missing. If you want to make other comments, hey, far be it for me to tell you not to write comments. But you don’t have to, you don’t have to, and relatively short assignments. You don’t have to, unless you want to. I mean, if you’ve got a student, they had all the elements in that, you want to draw a happy face, knock yourself out. [LAUGHTER]

John: Or at least for the artists, among us. [LAUGHTER]

Linda: Exactly. Now, if you want to go specs grading… because we’re talking about specs, we’re talking about specifications, like in a computer program, essentially… but you really have to look in the mirror and say, “What do I want my students to be able to demonstrate that they can do in this assignment, or on this test, or in this particular essay?” And that can be hard, because we’ve never had to be so specific, but this is what students love about specs grading. We get specific, we tell them what we want, they know what our expectations are, and they haven’t known, this might be the first time they’ve ever really known, what we wanted, because we have to be specific. And if we left out a specs that we wish we’d put in, well, better luck next semester, right? Just remember to put those in. But you can’t change those once you give students your list of specs. And then there’s bundling. With specs grading, you can also bundle assignments and tests together for various grades where students decide, “Oh, you know, you have to do so much work for an A, and it looks very difficult. I’m not going to go for an A. How about if I go for a B and that way this isn’t that important of a course to me. It’s alright.” And they might decide to go for a C. And you know what? what should you care? This is their decision, and if they’re going to be happy with a C, you can respect that. And otherwise, we look at C students like, well they were lazy and didn’t care and this and that. No, no, no. Students choose it for themselves. Fine. No problem. We don’t look down on our C students, as we unfortunately will kind of do sometimes. Anyway, but that specs grading. We’re on critical thinking and critical thinking, if you have the specs for something written, or a presentation for that matter, but if you know what you want students to show that they can do in terms of critical thinking, this is an easy way to grade, it really is. All the work is upfront.

Rebecca: Yep. It’s very time consuming upfront with what I’m doing right now. [LAUGHTER]

Linda: And it’s stressful. It’s stressful for us, because we “Oh, should I give this five points or six points?” Forget it. Don’t worry about that. And you know, students will go to you and argue for a half a point, they don’t mess with that with specs grading at all. They don’t do that. So it saves you a lot of wear and tear. And it’s better for your relationship with your students.

John: In large classes, how can you help students develop critical thinking skills if you don’t want to read lots of written work?

Linda: Ah, okay. Well, this is the beauty of groups. There are two methods that are particularly important in teaching critical thinking, there’s discussion, and that includes, by the way, debate…. debate’s really good, but discussion, debate. And you might have to do that in terms of just teaching in small groups. And that’s okay. Because the whole idea is that students are getting different points of view on a claim or a statement or something like that and then they have to defend it. But that’s all something that you can have students do in groups, those sorts of activities. Another thing that’s very important are some kinds of problems to solve. And I don’t mean the cookbook types. I’m talking about complex problems, fuzzy problems, if you will. And again, usually we have students work in groups for that, anyway. So that’s how you can teach it in large classes. It’s lovely if you have a small class, and everybody can hear everybody else’s statements and discussions and things like that. But if you don’t have that luxury, you can still teach critical thinking in large classes. And again, this is where spec grading can really, really shine. Because your grading time is cut to a fraction, an absolute fraction. Let me say one other thing. I talked about methods, but really the key to teaching critical thinking is questions, the questions that you ask your students, whether it be in class, or whether it be a writing assignment, or a paper or for a design or presentations, whatever it is, the key is questions. And each silo, each framework offers certain questions, that here you should have your students do this or answer these questions. So what I did was I put all of those together, synthesized them into a list of 45 critical thinking questions. Take your pick. Now, of course, you can look at your outcomes, and they make excellent assignments and questions and things like that. But these are general questions that make students think critically. And of course, you always follow up with “Well, gee, how did you come to that response? How did you arrive at that?” So anyway, there are certain questions that guarantee students are doing critical thinking and you have to adapt it to your particular class, but a lot of them, they go across the disciplines. And this is where you can see some of the similarities, like asking about the source of a claim, for instance, can you trust this source?”

Rebecca: How do we help students that come in with biases, preconceptions and things?

Linda: Oh, yes, absolutely.

Rebecca: …resisting facts. How do we start to overcome that?

Linda: Well, I think what you need to do is you start your course… you might have to start your course anyway… with selling critical thinking to your students. Because first of all, critical thinking is a skill that employers want. And I start my book with that, just so much evidence of that, say analytical thinking and that’s the same sort of thing. But there’s also beyond employment, there’s a quality of life. Now, if your students do not want to be suckers and fools, and if they don’t learn to think critically, that’s exactly what they’re going to be, suckers and fools. For instance, it takes critical thinking to avoid scams and shams. And if you google “scams,” you get billions of hits. It’s horrifying. Questioning and other things like they don’t know how to make sense out of popularized research studies. Well, a critical thinker knows enough to check those things. Advertising, that’s another place where you have to be very discriminating. And you have to critically think about “Well, what did that mean that such and such is twice as effective? Twice as effective as what? And what does ‘twice as effective’ mean?” There’s propaganda, fake news, disinformation, demagoguery, doublespeak, all this stuff to cover up the truth. And what students don’t know… a lot of people just don’t know… is there are fact-checking sites out there, there are quite a few of them. I talked about eight of them in my book, and how to get there and where it’s from, who puts this together. But it’s a good idea, I would say to give students an early exercise. And it could be, if you’re teaching chemistry, make it chemistry, if you’re teaching political science, make it political science, politics, to give them some fact-checking assignments. And if you’re worried about the politics, like all of these sites will call both sides of the political debate down on their lies or their distortion. So they’re fair that way. And so, if you want to get students to question their own politics, it’s great if you can get some good examples of things that they can go research where there’re lies on both sides, and that shows them that. But then they start realizing, “Oh, my god, there are lies. Oh, goodness, I’ve been believing all this.” There is Politifact.com. There is FactCheck. There is FlackCheck. There is OpenSecrets, Media Bias / Fact Check. There’s apnews.com/apfactcheck. There’s Snopes. There’s Truthorfiction.com. I mean, there’re all these sites. There’s so many of them, and that’s just the few that I talked about, because they tend to be better known. And they’re put out by like AP or something or Annenberg Foundation. And so that will teach them how much is garbage out there. Again, from both sides. Both sides in the political debate are full of garbage. But you also have debates going on in biology or in medicine. Remember, I’m sure you do, when they were saying, “Oh, coffee is terribly bad for you.” And then three years later, “Oh, it’s very good for you.” So it really hits all fields. Some of my very favorite ones is doublespeak. And I don’t know if students are aware of doublespeak like “servicing the target.” That’s military talk for bombing, to service a target. Yeah. “Neutralize” that means to kill. “Downsizing” means firing employees, “misconduct,” white-collar crimes committed by politicians, business leaders, military professionals, the police, that’s misconduct versus crime. “Detainee” is a prisoner of war. I mean, I could go on and on and on. And I do in my book.

John: You mentioned how important critical thinking skills are to employers. What does the evidence show about how effective colleges are in terms of helping students improve their critical thinking skills?

Linda: we’re not so hot, but it’s just put it that way. According to Academically Adrift, we improve the critical thinking skills of about one third of our students. Now, I don’t think that’s anything to celebrate myself. But I mean, at least we do that. But, these are often the students who came in with some critical thinking skills to begin with. The ones who need the help the most to critically think are not getting that help. They’re just not getting it. And so we’ve got to go out of our way to reach them. Again, most faculty think they’re teaching critical thinking when they’re not because they’re not consciously doing it. So there’s a sense in which we haven’t even scratched the surface of what we can teach in terms of critical thinking. And again, with all the different definitions out there, and all the different silos, I don’t blame faculty at all.

John: We can hope that your book will help reach more faculty and help us be more effective in this task.

Linda: Let me add one other thing that you can sell students on. Critical thinkers, and there’s research on this, tend to experience fewer difficulties in life than others do, whether they be health difficulties, financial, personal, legal, they just lead less troublesome lives. So if you kind of want to be happy, it’s a good idea to think critically, because it’ll keep you out of difficulty.

Rebecca: A good selling point, a good selling point.

Linda: Yes, it is a good selling point. [LAUGHTER] It is. And that’s just it. You might have to sell students on it. And you’re going to have to explain what critical thinking is to students, but they can take that pocket definition. It’s not negative thinking. It’s not just criticizing things. It’s nothing like that at all. And it’s not leftist, that’s not it at all. But they need that clarification because they’re coming in with misconceptions about what critical thinking is. And what the words sound like. Maybe it’s not a really good name for it, but that’s what we’ve been calling it. So we’re stuck with it.

Rebecca: It also seems like faculty make a lot of assumptions about students knowing how to analyze or how to interpret and what the question focus that you’ve indicated we should pay attention to, is really getting at that by asking the questions to get them to do the activity that we want them to do, when they might not know what that word actually means. Because it’s not been modeled.

Linda: Exactly, we should stop and model and then say, “You know, this is an analysis question that I’m asking you or that I just asked you.” We need to label things, so students get to understand what these cognitive operations are. And unfortunately, we make assumptions and studentsmight say, “Oh, analysis, that something a shrink does.” No, no, no, no, no, no. Interpretation. “Well, that’s something that people who read novels do.” No, we are all doing it all the time. So even chemists do interpretation. You interpret the data, and you can come up with different things.

Rebecca: And a lot of students get stuck right on describe, they don’t go much past that, and they don’t know that that’s what they’re doing

Linda: …or summarize. Yeah.

Rebecca: I’ve had conversations with students about those kinds of words before and it was really productive.

Linda: Yeah, yeah. because nobody’s really told them what these words mean.

John: It seems that higher ed is not doing quite as well as we’dlike in terms of increasing students critical thinking skills. Should this be an important focus for higher ed?

Linda: First of all, the buck stops with us. Otherwise, they go out into the world. And they could be adrift in general, without our teaching them critical thinking. They’re not likely to learn this in K through 12. I’m sorry. Oh, yes, they will in prep schools, but we don’t worry about those kids. So we have to keep people who are in power, and people with money honest, and the only people who really have ever wound up doing that are fairly well educated people. That’s why higher education is so important. And unfortunately, over the years… certainly, from when I was in college to now… the whole reason people go to college, and what they want out of college has changed. Now they want a job. Well, I hate to use this phrase. Back in my day, we wanted to develop a sound, sophisticated philosophy of life. Now that fits right in with critical thinking, critical thinking was very much a part of it. But now, that’s kind of like out the window. Well, to keep a job, you need to become a critical thinker. This is what employers need. This is what the world needs. And we can’t do good production without it. But even more important than that, we talk about freedom, for this society, any society, our freedom lies in our awareness of our patterns of human cognition, how they can be exploited, these patterns. And just, in general, our awareness of this worldwide pandemic of dishonesty in the pursuit of money and power, and it’s almost considered okay, because it’s all around us, we see it. Well guess what? It’s not okay unless we say it’s okay. So if we want to preserve freedom, we absolutely need this awareness. And it’s not like we’re doing something that hasn’t been done before. Because really, this awareness and this watchdog-type thinking really gave rise to a lot of successful movements, the environmental protection movement, the conservation movement, sustainability, recycling, stronger vehicle safety standards, and that’s just to name a few. And they all came out of critical thinking and awareness of where our weaknesses as human beings are. We all have a mind… well, almost all of us have a mind… and we need to know how it works so we can protect ourselves from all the advertising and scams and shams of demagoguery… which by the way, appeals to emotions… to all these different trips that individual institutions, organizations, corporations are trying to lay on us. And they are, because, hey, its profitable to lay the sawdust if we buy into it, we can’t afford to. We should not.

John: We always end with the question: What’s next?

Linda: Ah, yes. What’s next? Well, first of all, I hope getting back to travel.

Rebecca: Me too.

JON: Me too.

Linda: Yeah, exactly, exactly. Doing workshops and keynotes instead of this virtual nonsense where you’re talking into a screen. To see an audience tells me everything about what I need to do next in the workshop or in the keynote. But aside from that, I’m just finishing up another book: Online Teaching at its Best, the second edition. And in the second edition I talk about the kinds of teaching we’ve been doing lately, like remote teaching, hyflex teaching, Well, hybrid, we’ve been doing, but talking about it in another way. And in addition to the fully online learning that we talked about… Ludie Goodson and I talked about in the first edition… but the second edition has a lot more about what we’ve been doing lately because it’s possible universities and colleges might decide to keep some of this remote stuff. Hyflex is very much a disaster when you’re dealing with this pandemic and masks and all that. But, anyway, universities might say, “Hey, you know, it’s cheaper to do remote, we don’t have to build any more buildings, students can learn at home, hey, students might like that.” But again, they might not. But I’m afraid they’re going to do this with conferences, and it just makes me want to cry. Let’s do virtual conferences. We don’t have to pay hotel bills. And people don’t have to worry about getting travel funding. But in any case, it worries me that we’re going to get too accustomed to and accepting of, this virtual communication, which we all know, deep inside of our hearts, that’s not as good as human face-to-face contact. So anyway, but that’s next, that book will be coming out, I think, in the summer. And right now Ludie and I have the manuscript in but we’re still doing some pre-production stuff having to do with author queries, proofs, all that nonsense that makes you wish you didn’t write the book. [LAUGHTER] Writing is the fun part. This isn’t. But I’ll do it. I promise.

John: More information on effective online teaching, I think, is especially important these days. And I think we’re gonna see a lot more people doing it even when the pandemic is over, now that people have learned that it can work.

Linda: Yeah, it can work. But we just started doing these things, not knowing how to make it work. And learning has suffered, grades have suffered, and rigor has suffered. So far, no real good has come out of the switch. It can, it can, and that’s what the second edition will hopefully begin to do.

Rebecca: Well, thank you so much for joining us again, and having such a rich conversation.

Linda: Well, thank you for this opportunity. This has been great. Thank you.

John: And our faculty very much appreciates all the work you did with ACUE as well, because they keep bringing up some of the things they learned watching some of your videos in that course.

Linda: Oh, good. I’m glad to hear that. Well, thank you ever so much for this chance to talk about a book that isn’t even out yet but that I’m truly in love with.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

95. Specifications Grading

Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, Dr. Linda Nilson joins us to explore how specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

Linda is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Specifications Grading: Restoring rigor, motivating students, and saving faculty time as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

  • Linda Nilson – Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Nilson, L. “80. Self-Regulated Learning.” Tea for Teaching podcast. May 8, 2019.

Transcript

John: Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, we explore how the use of specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson. She is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning.

Rebecca: Welcome, Linda.

John: Welcome.

Linda: Thank you very much. It’s great to be here.

Rebecca: I think it’s more of a welcome back, right?

John: Yes. Welcome back.

Linda: Welcome back. Yes, yes, yes. Good to be here again.

Rebecca: Are you drinking any tea today?

Linda: As a matter of fact, I am. Yes, yes, yes. Of course I had coffee this morning as well. But I am drinking tea. I am drinking a berry tea, but it also has black tea in it. So it’s still a bit of a stimulant anyway, but it tastes really good.

Rebecca: Sounds good.

John: I have ginger peach green tea.

Rebecca: And I have English breakfast.

John: In an earlier podcast we talked with you about your work with self-regulated learning and one of the topics that came up with that was specifications grading.

Linda: Yes.

John: So we’d like you to tell us a little bit about your book, Specifications Grading: Restoring rigor, motivating students, and saving faculty time… especially that last part. [LAUGHTER]

Linda: Yes, saving faculty time. Yeah, I figured that would help sell the book. But it’s true. It does save faculty time. And one of the things that inspired this book was just my hearing so many complaints from so many faculty over the years, about grading and the aftermath of grading and returning grading material to students. And the constant steady stream of students trying to get another half a point, just arguing… just conflict… constant conflict… students being stressed… faculty being stressed… faculty getting larger and larger classes with less and less help… fewer and fewer TAs, if there ever any TAs. And I got tired of it. But there was a part of this that I did not invent. I heard it from a faculty member in the School of Management. And she was doing that pass-fail grading. It was saving her tons of time. She had huge classes, online classes. And she just invented this and she was also sick and tired… getting complaints from students and students not paying attention to her feedback, which of course, took her hours and hours and hours to write and return. So I took some ideas from her. But I also wanted to tie grades somehow to outcomes. And this is where another aspect of specs grading comes in. And that is with respect to bundling assignments, or turning them into modules or whatever. But I prefer the term bundles because it’s much more universal. So anyway, this was a solution to a problem. And that’s what a lot of my work has had to do with, making the faculty members job easier and more rewarding.

Rebecca: I think that you’ve been spying on me for the last many years. [LAUGHTER]

Linda: Yes I have. Yes I have. [LAUGHTER] I’ll admit it.

Rebecca: I knew it. I knew it. So I think John and I are both really interested in the idea of saving time, as are many faculty. But you also talked about, in your book, the history of grading… how its evolved… the 4.0 system. How is this different? And how does this relate to the history of grading?

Linda: Well, let’s look at the history of grading first. Grade started, well everywhere, in 1783. It was Yale’s idea. And what they started doing in that year was an achievement-based student classification system. They were not using As and Bs, what they were using were Latin designations of like optimi outstanding and pejores for failing, like as in pejorative, right? Anyway, then in 1800, Yale dropped the Latin designations and started using numbers 0 to 4. Sound familiar? But that was Yale doing that now. In 1850, the University of Michigan initiated grades, but for them, it was strictly pass/fail, and it only took 50% to pass. So we talk about grade inflation now… look backwards. [LAUGHTER] Mount Holyoke, though, just a few years later set passing at 75%. Harvard, also a little bit later, invented the A to F system, but passing was only 26%. So if you were wealthy enough, you had to know less, right? Okay. But anyway, that’s where ultimately grades came from before 1783. This was in Europe. And this started hundreds of years before our notion of grades. There was something… it was like a Jeopardy game, where graduates… or graduates to be… the hopeful graduates… were answering questions in a tournament style, but the stakes were really high. And so yes, if you are winning throughout, you were really showered with honors. But if you were at the bottom, you lived years in shame. It was terrible. We talk about high stakes. Oh my. So anyway, grades were invented after universities. Socrates didn’t talk about grades, right? It’s a relatively new invention for sorting students.

Rebecca: How does the specification grading relate to this letter system or this 4.0 system?

Linda: Well, it kind of takes a break away. Because first of all, all this grades that I was talking about was with respect to courses, there have been pass/fail courses. Sometimes they worked well, sometimes they didn’t work so well. They work pretty well in medical school ‘cause we’re dealing with a highly motivated students who really understand the need to know. But other than that, most students would do the absolute minimum to get their C-… whatever, it didn’t work very well in terms of like motivating students to learn, they learned the absolute minimum. With specs grading, the pass/fail is within the course, the assignments and tests in the course. But, you don’t pass with a C or C-. You set the passing level at what you would regard as a B level. And this is what restores the rigor. I think we have been sold a poor bill of goods, when we don’t set our students to high standards. We say, “Well, maybe they can’t do it, and then they’ll get…” They can do it. Come on. They’re just not doing it because of our partial credit system and our point system ‘cause they can always get another half a point, right? …just by wearing you down. [LAUGHTER]

Linda: Really, they do and they know it works, it worked in high school, so it works now. Now it may have worked the previous semester, these students aren’t stupid. And they can do what we ask them to do for “B” work… certainly at the undergraduate level, or actually at the graduate level as well. They could do it. But they choose not to, because they can survive otherwise. They could do well enough to pass or get their “B” or even get their “A.” So why should they sweat it, they can always get the partial credit. And students get partial credit. It’s almost like going back to the University of Michigan 50% passing for assignments and tests. I mean, we pretty much lowered the bar, because we want so many people to get over it. Well, people are getting over it without preparing… without doing a decent job. And they’ll spend 10 minutes… 20 minutes… the night before on an assignment knowing that no matter what they do, they will get partial credit and they will pass. So again, they’re not stupid, it’s just that, for them, college is a game that doesn’t have much to do with learning. That’s not where the focus is, and at least with specs grading, there is a lot more focus on learning, because you are tying the assignments and tests to student learning outcomes. And that’s a really nice part of the system in that those grades, the ABC, mean something in terms of outcomes achievement, and these, in turn, might be tied to program outcomes as well. So all of a sudden, you’ve gotten rid of an entire step at the departmental level of having to measure program outcomes. And why do you have to do this? Because accrediting agencies know that our grades don’t mean much of anything, as far as learning is concerned, as far as what they’re interested in, which is outcomes achievement. And outcomes achievement, it’s an up-or-out sort of thing. You can either do something or you can’t, and as long as there are standards set for that, and certainly the accrediting agencies want standards set for that. So let’s say “Okay, so your goal is a student can write a good quality, maybe not great quality, but a good quality business proposal.” Okay, fine. So if that is your outcome, what really does that entail? What exactly are you looking for in terms of a very good business plan? And that should be incorporated in your directions to students. But we don’t articulate that, do we? We really don’t. We speak very vaguely, “Well a business plan should have this and here’s your rubric.” Well, first of all, students often don’t understand the language, but is simply not detailed enough such that all students can understand the directions and actually follow them. We need to put more detail. Usually what we assigned undergraduates is some sort of a template or a formula to do this. And we’re not talking about that. We’re not sharing this template or formula with our students. And that’s what we need to do… not to say that you can’t allow for creative work. Matter of fact, with this system, you can and you don’t have to worry about tearing your hair out in different ways. “Well, I don’t know how to grade a movie.” Well, no, but you can talk about certain qualities of that movie, as in just simple things like the length, or perhaps the number of scenes that you want to see. And certainly they’ll be a learning goal and a communication goal connected to it. And that’s pretty much all you have to do. For most of our assignments, they are formulaic, but students don’t know the formula. And we need to tell them, and those are what you would call our specs for an assignment. So that’s really what we need to do. So all our work is up front and laying out what those specs are. It’s like a one-level rubric. But we don’t have to worry about different levels of the rubric, where there are four-level rubrics and five-level rubrics. We don’t have to worry about that. All we need is one. So we can afford the time to actually specify what we’re looking for… what that template involves.

John: And then the focus is helping students reach the standard, rather than negotiating with them over the grades for what they’ve received.

Linda: Exactly, exactly. Yeah, forget negotiation. There’s some things that ought to be non-negotiable. I mean, we’re not sloppy when we grade, I know, we get sick and tired of it. But we are so diligent in our grading. And then we have a line of students outside our office saying, “I deserve another two points on this because Susie said the same thing. She got 12 points, I only got 10.” …like write an essay about it… justifying it.

John: One thing I really like about it, though, is that right now we have these two levels of assessing students’ learning, we have these complex assessment plans for each department. And then we have a grading system, which often bears little resemblance to the assessment. And this is making the assessment transparent and obvious to students. And it’s forcing both faculty and students to focus on the learning objectives for the course.

Linda: Yes, as we should be, you think about it, what does it mean, when a student gets an A? Does that mean that that student has achieved all the objectives? all the outcomes at the level we want to see? Well, maybe for some of the students who get an A. What does a “B” mean? Now we’re starting to get into really ambiguous territory. Does that mean that the students sort of achieved or barely achieved all the outcomes or maybe achieved some really well and then didn’t achieve others? And a “C,” forget it. You can’t tell what’s going on there, especially given the way we give out “Cs” these days. So maybe the “C” student achieved one outcome well, but which one? Yeah, I mean, no wonder accrediting agencies pay no attention to our grades. They don’t even want to know them. And I can understand why, if they’re focused on learning outcomes. And so yeah, we’ve got this whole extra level of work that we have to do and the department chairs have to do and the Provosts have to do. I mean, who needs it?

Rebecca: Can you talk a little bit about the relationship between specs grading and contract grading, because it seems like they have some things in common?

Linda: Yes, they do have something in common. With contract grading, students individually work it out with a faculty member as to exactly what they’re going to do. And if they do all those things that they will get there “A” because I’ve never heard of a student contracting or a ”B,” it just doesn’t happen. Now, contract rating goes back to the 60s, and the first half of the 70s. And what was happening was since there was this individual relationship, first of all, faculty didn’t specify enough about what they wanted… what constituted “A” work… same problem we have had up to now… and faculty weren’t all that much different then, in fact, they probably gave less guidance. But in any case, there were the specifications. And then faculty and students would develop this relationship, this individual relationship. And it became particularly difficult for the faculty member to give anything but an “A” to the student. And the student would have thought, “Well, why didn’t you tell me to do this to get my ‘A,’ because that’s what I contracted for.” So contract rating was: 1. sloppy, but 2. highly individualized. With our class sizes, we can’t even talk about contract grading. And because it was something that was mutually established, between the faculty and the students, and specs grading is nothing like that. This is all in the faculty members’ hands. Well, at least, setting out what is required to get the “As” and “Bs” and “Cs” and “Ds” . But students choose what grade they’re going to go for. And according to what grade they believe that they need, but also according to what kind of a workload they want to shoulder for this particular course. And you know, maybe all they need is a “C.” And you know what, that’s okay. And we don’t look down on that student if the student says: “Well, all I need is a ‘C.’ We do otherwise. We say “Wow, this is a lazy student.” That’s what we’re thinking in the back of our heads. We have negative thoughts about students who settle for low grades. But with specs grading, you don’t have to feel that way. This student chose a seat for whatever reason, I don’t care. My course is not in that student’s major. So I’m not going to take it personally. So, that’s nice. But the fact that that’s making that choice, gets rid of all kinds of grading complaints, or that thing at the end of the semester: “What can I do to bring up my grade?” Well, it’s very clear what you could have been doing. If you went for a “C” now you could have done these additional assignments, or taking this additional test to get a “B.” But you didn’t do that. Now, you’ve got a week left, maybe you want to try to do that. But don’t ask me, I laid out the contract, I laid out the terms, I laid out the specs at the beginning of the semester, and you make your choices. This whole thing makes students feel a lot more responsible for their grades. They made the choice. And we respect that.

Rebecca: So I think a question that would come up for many faculty is you’ve laid out that certain assignments need to be accomplished to get even a “C…”

Linda: Yes.

Rebecca: …and what the specifications are. What happens when a student struggles to meet those specs?

Linda: Oh gee. Well, now when you say struggle, you can struggle before the assignment, or you can be disappointed and angry after the assignment. So if you’re talking about struggling before and say, “Professor, I don’t understand these two specs. I don’t even know if I can do it. Could you give me some more guidance?” No problem, and then because the stakes are higher, you get credit or no credit whatsoever… doesn’t count towards your bundle… doesn’t count towards whatever grade you’re doing. So you’d better figure this out. Now, afterwards, don’t meet the specs. Okay. And obviously, you didn’t come to see me in advance, did you or otherwise, we would have hashed this out. Anyway… so you didn’t pass? So you missed these two specs. All right, well gee, that’s most unfortunate. But there’s this system called tokens… or I call them tokens, but you can call them… I don’t know, you can call them pigs in a blanket for all I care, it doesn’t matter. They’re like get out of jail free cards. I’ve seen them called hail Mary cards. In a geography course, they were called globes, but anyway, they are opportunities to either redo an assignment that didn’t meet the specs, or to get a 24-hour extension. And there might be some other things that you make up along the way if absence counts against you in your course, you can get out of absence. But let’s say it’s for redoing this assignment, which is really important early in the semester, because students are not going to believe that you’re going to grade them pass/fail, because this is all new for them. And they go, “Well, how can this be…” and “There must be some sort of partial credit or something…” They won’t believe you. So there will be a number of students who will need this token or get out of jail free card to redo the assignment. It’s up to you, as a faculty member, as to how many get out of jail free cards or tokens that they might have. Three is a nice number. I like it, but some people give five. Some people give two. But I think it behooves you to have some reasonable number of tokens with specs grading, because that takes the pressure off of students; it allows them to screw up at least a few times. So this is something that students can get second chances out of this because the stakes are higher than typical. But the thing is, when you’re grading these assignments, all you’ve got to do is check the specs that weren’t met, and say, “This is why your assignment failed.” Now, if you want to give additional feedback, by all means, I’m not going to stop you. But I want you to have a life. That’s the only thing. And you can give positive feedback as well. And then you know what, you’ve passed something, then you get positive feedback from your instructor that you figure, “Wow, the instructor gave me this positive feedback out of the kindness of her heart, because she really cares about me and my success… told me how I could maybe do better in the future.” First of all, I’m going to read this, because this is meaningful feedback. This is not justification for having taken off points, because that’s what most of our feedback is about: why you didn’t get full credit. And now this is actual substantive learning feedback. So, it decouples feedback from evaluation in a way, but students get the chance to do it over and now that you have their attention, that let’s say they didn’t pass a certain assignment, that they can come to you and say, “Okay, how should I do this better?” Oh, okay. Isn’t this lovely? So the worst that will happen is you’ll have a conversation with a student about how to improve his or her work, that’s the worst that will happen. Isn’t that wonderful? [LAUGHTER] And sometimes students know, they just got lazy.

Rebecca: How would you suggest structuring specification grading for a class that’s more project based or even collaborative work?

Linda: Oh, sure. Whether it’s projects, or papers or whatever, again, it’s a matter of specs. Oftentimes, with our projects, that we might allow our projects different media, what we have to do is lay out specs for each media. Or say, if you can think of a different media, you can do a film, you can do a play, you can write pamphlets, you can write a paper if you want to, or you can do any of, let’s say, a half a dozen things. And let’s say if you want to do something else that’s not on the list, come talk to me, and we’ll work it out, we’ll work out some specs for it, too. So what you have to do is you have to lay out specs, but oftentimes the specs that you have to lay out, and students love creative assignments, is you have to lay out, let’s say, length, and so you might say, “Okay, if you’re going to go with a film or video, I want 20-mintues of that. Going to go with pamphlets? I want at least four pamphlets, I want each pamphlet to have at least 250-words in it. If you’re going to go with writing a play and performing a play for your peers, I mean, wow. But I do want it to be a half an hour long. And I want it to involve everybody in the group, everybody’s acting. And if you’re going to write a paper, I want it to be at least, let’s say, 2000-words long… or maybe that’s too long, whatever… and you lay it out, and I want it to follow this kind of an outline.” You can lay out specs for different kinds of media, and let students run with the project. Now a design project, what you might allow students to choose is exactly, let’s say, in architecture, what kind of a building they’re going to design. Let’s say, “Okay, you can design a residence, single family, a separate dwelling, but I want it to be at least four bedrooms, I want to be two stories, this and that. And you could do an office building too. But I want this to be at least two stories, and I want it to be so many thousands of square feet.” And then you let them run with it. And you know what, we know from our history, and from a lot of publications, that students run with this sort of thing. So I’ll give a little guidance and off they go, it’s a beautiful thing. Now in terms of projects having to do with group work, I don’t know that that specs grading will make group work smoother. [LAUGHTER] And I’ve talking about out-of-class group work. If you want to play marriage counselor, you just go right on ahead. But you don’t have to. I mean, there are some people who say, “Hey, you guys work it out on your own, I don’t want to hear about it and you can fire freeloaders. And if you are doing all the work, you can resign from your group and any other group will be happy to pick you up. But all this has to be done by the end of the fifth week,” whatever. So I don’t know that we’re going to get rid of those problems. [LAUGHTER]

REBBECCA: I think my question was more about the high-stakes nature or the long-term nature of projects.

Linda: Yes.

Rebecca: So would you recommend doing scaffolded specs or something like that to help make sure they’re on track?

Linda: Well, yes, what you need to do, as with any major project, you need to divide it up into smaller tasks that students can be at accountable for along the way, and maybe even get feedback for. Now, you can give feedback… your peers… people in other groups… can give feedback. Because after all, we’re doing criteria referenced grading, we’re not grading on the curve. So there are different ways for students to get feedback. But in any case, this divides a massive task into little pieces. So maybe starting out with, “Okay, by the end of the third week, we want your group to have a literature review put together.” And we put specs for the literature review. “And we want so many references. And we want so many of them in the last five-years,” And what have you. So we can even put the specs, I even specify specs for the format that we want: APA format, or ASA format, or Chicago, whatever it is that we want. That’s something that we can put in the specs if we want to, and that way students will actually proofread. Now I think we ought to be somewhat tolerant with respect to format anyway, we ought to allow three little errors in format. Because you know what, when we submit a paper to a journal, they’re tolerant of us, right? [LAUGHTER] I mean, you can’t be sloppy, but if we put a period instead of a comma, hey, we are forgiven. And we’ll catch it later on the proofs. So we need to allow some errors on something like that. But anyway, yeah, we divide up a big task into pieces. And then students get feedback on the pieces. And that’s also a good way to find out early about who are your freeloaders, who’s not doing the work early, so they can get rid of the freeloaders if they need to. Here’s another thing that you can do with specs grading, when you do bundles, you know, the project, the big project, you can make that big project required for an A, period. And then you will only have groups with students who are going for As. So it depends on the size of your class, of course. You might wind up with only, let’s say, three groups who are going for As. Or you know, the number of students that will fall into three groups. Well, guess what, you’re going to get some really fine projects, you’re not going to have to worry about it, and they might not be freeloaders. But anyway, in the examples in my book that a lot of faculty members and I recommended as well, that group projects be reserved for getting an A, for the A bundle. And other assignments be in the B bundle and C and the D bundle for that matter, which might be very minimal. But hey, if you want to a D, far be it for me to stop you. So, anyway, that’s another way to solve the problem, you just put those projects in just for the students are going for an A.

John: You’ve mentioned bundling a few times in terms of grades, could you talk a little bit about that and how you can go from the specs grading to the course grades that are assigned in the course.

Linda: Okay, there are two ways and the first way I’m going to just mention and not recommend. But you can keep your point system, and you might have some haggling over grades. But in any case, you can keep the point system and say, “Okay, my course is 100 points and if you get 91 points or more, you’ll get an A.” And okay, fine, but the bundling system allows students to choose their grade. So what you do is, you set up clusters of assignments and tests that students have to pass at that B level to get credit for their bundle. So you set those up. And the easiest way to do this, so you got a D bundle, is pretty minimal. And you can tell students, “Look for your D, here are the only outcomes that you will be able to achieve if you go for the D.” Some students say, “Hey, I don’t care, not in my major.” Fine. So it’s minimal work, maybe it involved just passing tests at let’s say, a 70% level, and maybe doing a couple of little written assignments along the way. But, they all have to meet whatever the specs might be. Whatever specs you set out for those assignments… And C bundle, “Okay, you got to do everything that you do in that D bundle, plus there are extra assignments as well.” And so maybe what you are having students do, and I’m just making this up as an example, is the students have to turn in 15 out of the 18 reading assignments that you have, they have to turn in a typed up outline on a particular reading. This way, at least your C students are going to do the readings, right? Because at least most of the readings, almost all the readings, and you’d give your teeth for almost all your students to do almost all the readings, right? But that’s all they’ve got to do. Maybe for a B you have to do all the requirements for a C, maybe even more of those reading notes, or something different. Perhaps keep learning journal, keep a journal on how you are learning, what you’re having trouble learning, the different strategies you are trying to learn the material, and it’s going to be collected. But you really have to specify what questions you want students to answer. So that will be collected maybe four times during the semester, and you might collect some every week or so and look at a few of them. But again, you’re just looking for the answers to the questions. That’s what you have to do for a B. You’re going to get a good handle on those readings and you’re going to find out how to best learn this material. For the A, you got to do everything for the B, but you have to do some sort of a group project where you’re going to learn, let’s say application, analysis, evaluation, what have you, whatever higher thinking levels that you have in there. At the C level, that’s pretty low. But good lord, at least they’re understanding the readings, at least they’re reading the readings and presumably understanding it. And you know, you can spot check some of them to see that they are understanding and if they’re not, you can help them with that. And as soon as they’re learning how to learn for that B, but students are learning how to analyze the material, even higher-level learning outcomes for that A. Now if your student learning outcomes are dictated by a professional accrediting agency, then you’ve got to put all of those outcomes in what’s required for a C, or maybe your institution considers D passing, some institutions don’t. Whatever you consider passing, that’s where all of those outcomes have to go. B students and the A students, they will achieve even more outcomes. If you’ve got an outside accrediting agency telling you what your students have to be able to do, then guess what? All the students that pass the course have to be able to do these things. And you’ve got to set out the specs accordingly. There are some accrediting agencies that layout good outcomes with active verbs. And other accrediting agencies, they give you crummy outcomes where you’ve got to essentially rewrite them so you can assess them. But I’m not going to get into that. [LAUGHTER] Did I answer your question, I hope?

Rebecca: Yeah, I think a lot of your examples focus on a more traditional class and not necessarily a lab or a studio kind of class where there’s a lot of class work that happens inside of class. What might a bundle look like in a situation like that? Where a lot of the learning activities are happening in person rather than for homework. So, for example, I teach studio classes that meet for six hours a week, and they presumably do about three hours outside of class. Versus the reverse, which many people have and labs are similar.

Linda: What area are you in?

Rebecca: I teach graphic design.

Linda: Okay.

Rebecca: But it would be similar to something that would happen in the sciences too and having to do lab.

Linda: Yeah, to an extent, it depends on whether it’s a decent lab or not. [LAUGHTER] Some labs are a waste of everybody’s time. But anyway, what you have to do there is, you’ve got projects, right? …throughout the semester, different things that students are doing. but how many projects do you have in a semester?

Rebecca: Usually, it’s like four big projects, or three.

Linda: Okay, well, now this sounds very, very radical. But what if for a D, you only had to do one project? …and I mean, at that B-level that you set to pass. What if for a C, as a student, I only have to do two. Now, for the first two, because I presume this is somewhat cumulative. For a B, I only have to do three projects. But an A, of course, I have to do all four, if there are four projects. Now, what does this mean? Oh, by the way, do you have tests?

Rebecca: No.

Linda: Just curious. Okay, it means that assuming your D student does passable work, that first project, you can say bye bye after that first project is done. If the person wants to sit in… you can’t, you know… it’s a free country, right? Well, not really. But they can sit in if you want them to, that’s fine. For a C student again, you can say bye bye after the second project, assuming they do it at that passable level. And by the end of the semester, you have only you the most motivated and committed students that you have to be concerned with. And you can give them some very challenging work. Now, I don’t know how you feel about that. But that’s the way you could do that. Now, you could also bundle it differently. You can expand the number of projects that you have as well and simply make them shorter if that’s what you would be required. But I don’t know how you react to that.

Rebecca:I think my accrediting agency would react to that in that they have to put a certain number of hours in. [LAUGHTER]

Linda: Okay, okay, so your accrediting agency is time focused.

Rebecca: It’s part of it. Because presumably with more practice, you get better.

Linda: Well, presumably. [LAIUGHTER] …practice with feedback, you get better. But yeah, if we are committed to this hours business, that makes things very unruly, everybody’s got to put in the same amount of time…

Rebecca: In class anyway.

Linda: Oh, in class, okay. That’s very unruly, with respects to specs grading because specs grading isn’t about time. I mean, you can recommend that such and such assignment should take you at least three hours for it to be passable. But other than that, that’s your recommendation for a certain given assignment. But yeah, that makes it very unruly when your accrediting agency says, “Okay, they’ve got to spend so many hours in class and not just looking at their phone.”

Rebecca: Yeah, I mean, we do critiques and things like that. We provide a lot of feedback. But yeah.

Linda: Yeah. Okay. You do critiques?

Rebecca: Yeah lots of critiques.

Linda: Yeah, like art classes and things like that?

Rebecca: Yeah.

Linda: Okay. Well, that’s nice. But now, you could hold your B and C students to doing critiques on all the assignments, just so they can put in their hours. But they don’t necessarily have to do the assignments, at least they can see models. But again, I don’t know if your accrediting agency would be happy with that either.

John: This may not exactly be specs grading, but could you give, say, different levels of activity in each of the assignments? If you have four-projects, you could have one bundle in that project that would give a C, another that would give a B, and another that would give an A, changing the scope of the projects and it would be specs grading within that scope.

Linda: Yes, that has been done and I’m not sure about what kind of projects you’re doing. But yes, you could do that. And you could set different specs for each level. So D students, lets say, have to do a D-level project, which is not nearly as time consuming and not nearly as high a level cognitively, it doesn’t demand that much, doesn’t demand much time, it doesn’t demand that much thinking. And so you could do that and that way students would put in their time, and as long as you can set out the specs for those levels. Yes, absolutely you could do it that way.

Rebecca: Yeah.

Linda: But yeah, accrediting agencies can really get in the way of flexibility. Rebecca, would your accrediting agency go along with that?

Rebecca: With scope, probably. Yeah.

Linda: Yeah. Okay, good. I want to tell you about a course that does this, it’s in computer science, they have students writing programs, and there are six projects that students have to do, they do these individually. If you are going for a C, you get some pretty easy projects to do, pretty easy problems to solve. If you’re going for a B, well you get sort of intermediate-level problem, you don’t do the C one. Because if can do the B ones, you can do the C ones. Now for an A you get some much more sophisticated ones. But still, for every level, you only get six. So there is that, that as long as you can designate level of difficulty, or for that matter, breadth of knowledge, if you can designate that… of course, usually people can do that… that you can make levels out that you can make different bundles out of that. This is another little interesting take, in this particular course, if you are late handing in your projects, that you’ve got to do two more projects at that level. So in other words, you are penalized for lateness by having to do more work. Now that really hits students where it hurts. And if you’re like super late, you got to do yet two more. So any way that keeps students on their toes. [LAUGHTER] Anyway, so in this particular course students are rarely late. So yes, there are ways that you can do that just in terms of level of complexity or level of cognitive operations demanded.

John: How do you explain this to students, if you’re going to introduce a specs grading system?

Linda: First of all, you’ve got to sell specs grading, because they’ll look at you like, “Huh?,” which is fine. And you should explain this, but you can tell them, “You know, I’m going to hold you to higher expectations.” And we know that holding students to higher expectations feeds into student success, student learning, but also student success in general. You could explain to them the concept of andragogy. In other words, that pedagogy really has to do with little kids. Andragogy has to do with adults. Saying, “I’m going to treat you like an adult.” And students like that kind of thing, you might be choosing the kind of project you’re doing. It’s easy to build choices into this. “This will be a safe but a challenging environment because you know exactly what you’re supposed to do. I’m going to give you the descriptions of what you have to do for the assignment to pass in advance and in detail. So you almost can’t screw this up unless you don’t read the directions or pay attention to the directions.” So you want to emphasize the choice and control over their grade that they’re going to have. You want to tell them about tokens, we’ve got some wiggle room, if you will. Now these tokens, by the way, are not physical, you make them physical students will develop a black market, [LAUGHTER] believe me, but virtual and you keep track, or you can ask for an extension, but you don’t ask me for an extension. You’re late, I just take a token away. That’s all. We don’t have to talk about it. I don’t care why you’re late, this is just the way it’s going to be. Just like your boss won’t care why you are late or why you are absent, it counts as like a holiday. Now we’re going to tie what your grades is to what you are learning. If they look at the syllabus at all, everybody’s going to know what you have learned in this course. So that’s how you sell it to students. Now students will need to be reminded about how the system works, because this will strike them. “This is so weird.” Even if they initially like the idea, they still need to be reminded a few times because it is really strange. According to my research, students way prefer this system to traditional grading. And one of the reasons why they do is because we give better direction, we tell them what we want, we give them the formula, we give them the template. or we give them tons and tons of freedom to meet the specs. Especially with students when we talk about length or whatever, length means depth to the students. It doesn’t mean that necessarily to us. Students really like this because they feel way more secure in it. And they do like the element of choices that are built in, that’s motivating for them. And they do learn more. And if they are more motivated, they’re likely to do more of the work at a higher level, they are more likely to excel. And the A students are going to do A work anyway, because they don’t know any better. All they know is A work. And there’s a sense in which they want us to love them. I mean, really, so they’ve got this strong sense of loyalty towards us and they really want us to respect them. And they’re used to feeding off of that they’ve been doing it all their lives. Don’t worry about them getting sloppy, they’ll never get sloppy on you. [LAUGHTER]

John: If someone wanted to transition to specs grading, how should they get started?

Linda: What you might want to do if you’re switching, transitioning to specs grading, you want to look at your assignments, and for that matter, look at your tests, like what you have in your tests. You want to look to see what can be transferred into pass/fail. You also want to look at your tests, if your tests are very objective, and you’re relying on a test bank, remember, it’s so much easier for you to grade with the specs that you can start assigning more written work or design work on tests, what have you. But definitely higher-order thinking types of questions and you will have the time to grade them, because you’re going to lay out the specs and without giving away the farm, you will tell your students the specs of your essay questions in advance, and they will study accordingly. But anyway, you want to look at your outcomes and then you want to identify the cognitive level of your assignments and your test questions which again, you might want to change. You want to be able to group your assignments and tests by that cognitive level so you can develop bundles. It’s kind of a radical way to do it, but it makes your life so much easier once you have those bundles. Because all you’ve got to do is say, “Okay, let’s see. So I had four assignments in this bundle, this student passed them all, hey B. It’s so much easier, you just have to be sure that you set out the deadlines for the different assignments in the bundles. You don’t want everything being submitted to you in the last week or two. So you still have to have your deadlines along the way. Just warning you about that. I want to refer people to my book for examples of courses, and not just my courses either, that have been specs graded. And they cover a range of over a dozen different disciplines. So they can see all kinds of examples.

John: That’s a great resource for those who are considering moving to specs grading.

Linda: Yes.

Rebecca: And it also sounded to me like a couple of cautionary tales that you have about transitioning are about time and making sure that you spread things out…

Linda: Yes.

Rebecca: …but also being very clear in what the specs are, and then following through with whatever you said.

Linda: Yep, yep. And if you didn’t lay out enough detail, well, there’s always next semester. [LAUGHTER] But you can’t change those specs in midstream. “Oh, I wish I thought of this and then start grading students.” No, no, no, no, no. But again, we live and learn, right? We’ll get better the second time.

John: We always end with the question. What are you doing next?

Linda: Oh, what am I doing next? Well, what I’ve been doing is I’ve been doing a lot of traveling, doing keynotes and workshops… oftentimes on self-regulated learning, sometimes on specs grading, and sometimes on any number of different topics having to do with either teaching or academic writing, or something like that. So I’m going to be taking a little bit of a break from that until the end of, or late, July anyway and then things start up again. But that’s kind of nice. I don’t mind that because this keeps me semi-retired and that’s the way I want to be. A semi-retirement is Nirvana, just letting y’all know. [LAUGHTER] So, that’s what’s next for me, that and to finish my tea.

Rebecca: Sounds like the key to semi-retirement is the semi not retired part. [LAUGHTER]

Linda: Yes, it is, it is. Your brain is still working and you still got your thumb in the pie so to speak. And it feels good because you’re not under that same 60 hour a week pressure that you otherwise have with a regular job and then doing these other extra things on the side and it doesn’t work real well.

Rebecca: Thank you so much for joining us.

Linda: Well, thank you for this opportunity. I normally say… autograph books and things like that… happy teaching, but I want to wish everybody happy grading. [LAUGHTER] An odd phrase right? Happy grading. [LAUGHTER] What an odd phrase, right? Happy grading!

John: Thank you.

Rebecca: Thanks, Linda.

Linda: Bye bye.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

80. Self-Regulated Learning

Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, Dr. Linda Nilson joins us to examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

Dr. Nilson is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning. In this episode we focus on discussing one of her books: Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills.

Show Notes

  • Linda Nilson—Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2013). Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-awareness and Learning Skills. Stylus Publishing, LLC..
  • Nilson, L. (2014). Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. Stylus Publishing, LLC.
  • Professional and Organizational Development (POD)—Network in Higher Education Conference
  • Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6), 1121.

Transcript

This transcript has been edited to improve readability.

Rebecca: Most students arrive at college with serious misconceptions about effective learning strategies. In this episode, we examine what we as faculty can do to help students develop their metacognitive skills and become self-regulated learners.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson, the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. Dr. Nilson is the author of many superb books, book chapters, and articles on teaching and learning. Welcome.

Linda: I’m very honored to be here. Thank you.

Rebecca: Today’s teas are…

John: Are you drinking any tea?

Linda: Yes, I am drinking tea. I am drinking Lemon Lift.

Rebecca: Oh that sounds like a great way to start the day.

Linda: It is. It’s a very good way. Well, I also started it with coffee, but… [LAUGHTER]

John: And I’m drinking Ginger Peach Black tea.

Rebecca: And I have my Golden Monkey tea today.

John: We’ve invited you here today to talk about your book, Creating Self-regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills. Could you define what it means to be a self-regulated learner?

Linda: Yes. Self-regulated learning is the conscious planning, monitoring, and evaluation of one’s learning for the purpose of maximizing it. That’s a very fancy way of putting it. It’s that voice in your head that asks you questions about your learning as you’re involved in some sort of learning task, questions like, “Okay, I’m going to do a reading now, what strategy works best for me?” Now you just might brush over that because you’ve done readings of this type a dozen times, a hundred times, whatever, but you’ve asked yourself that question along the way. “What’s my best strategy? What kind of a task is this? And monitoring: are my strategies working for me? Am I getting it? Can I paraphrase the last couple of paragraphs that I just read?” It’s a reading thing, but it works also in lecture. And then at the end, you evaluate yourself. “Well, let’s see. I had a goal, being able to recite five main points from this chapter, let’s see if I can do it,” without looking at the chapter of course. [LAUGHTER] So you evaluate your abilities, you evaluate your strategies. That’s really what it’s all about and it involves a great deal of talking to yourself.

Rebecca: So how did you get interested in talking to yourself? [LAUGHTER]

Linda: Well, I heard voices. [LAUGHTER] Well, how did I get interested in this topic? Actually, it was an accounting professor at Clemson that got me interested in it. This is 2007, right, this is a long time ago. And so she said to me, “What about giving a workshop on self-regulated learning?” In my head I said, “Huh? What’s that?” I’d never heard of it. And so I decided to go find out about it and it took me a few years to really get a workshop together on it and I decided, “Gosh, this is wonderful. This is learning how to learn. This is familiar to me,” because I’ve been talking to myself for years. [LAUGHTER] So I thought, “Okay, I’m not crazy. This is a learning strategy, a major learning strategy, one that you can use throughout your life.” And so I gave the workshop, I started giving workshops, like at the POD Network Conference—which is made up by people like me who go to this conference every year—and then I decided—well, I didn’t decide—a book publisher came up to me and said, “Please write a book on this, I will publish it.” Since I was in love with the topic anyway, I decided to do it. And so I did and delved into it deeply.

John: As you’ve described it, it sounds like part of this deals with improving student metacognition, but you note that it goes a bit further. Could you talk about the additional aspects of it?

Linda: Metacognition is the cognitive part of self-regulated learning, which is a major part of it. However, there are a couple of other elements to it that I don’t know that you could say are really focused on cognition. There is the emotional element to it, which involves getting yourself to be motivated and interested in the topic. Remembering, reviewing what your professor told you about the relevance of this topic, and thinking about it yourself. We can motivate ourselves, we can reframe a task for ourselves, and we can certainly reframe what is going on in terms of a learning experience. That’s a major, major part of it. The emotional part of the end is: “If you didn’t reach your goal, what do you do about it?” Do you give up, walk away, and say, “Well, I wasn’t born to do engineering,” or whatever the topic is. No, what you should say is, “Let’s try another strategy. Let’s look into possible strategies.” As instructors, we need to familiarize students with various strategies because they come to us—I like the phrase—“as feral children” in terms of the life of the mind and what they know about learning. We don’t have cognitive psychologists—unfortunately—teaching first grade or fifth grade, and so we need to equip them with how their mind works. There is one other element, a physical element, and that involves planning, monitoring, and evaluating your physical setting, where do you study best. If doing a reading or writing assignment, is it in a coffee shop, or do you have to essentially be in a soundproof booth where you don’t have any stimulation? How much coffee should you have? Or tea? [LAUGHTER] What kind of an environment should you set up for yourself—perhaps putting your digital distractions in another room. How should you schedule your breaks? Other things that you might want to consider is the amount of sleep that you have had because that can be a very important element of learning and writing. Some people study better to mild background noises as long as they are familiar music they’ve never heard before. You’ve got to try out these different things and find out your best setting.

John: In your book you describe how you became a self-regulated learner. Could you relay that story?

Linda: Yes, it was based on fear and terror. [LAUGHTER] As a child, I went to a private Catholic girls school—great education, but not in the sweetest of ways. From about fifth grade on, we had what was called recitation every single day in English and history classes. The nuns would ask a question and would randomly call out students’ names—we were in small classes so it wasn’t an absurd thing—and we had to get up and we’d better have the answer to the question. Now, not all the kids did, but I needed to be Little Miss Perfect because I wanted to get into college. Somehow I thought that the answers that I gave in fifth and sixth and seventh and eighth grade would or would not get me into college. So I learned to quiz myself while I was reading chapters or essays. Constantly quizzing myself. And another thing we had to do, usually later in the day, was called “exercise period.” We had 35 minutes to write an answer to an essay question that was generally related to the readings. And we couldn’t look at our books. It was out of desperation that I was trying different strategies so I could perform very well every day, and I was really quite successful. So when I was reading about self-regulated learning, I said, “My God, I was doing this as a child.” That’s why it sounded so familiar to me.

John: But that’s not the experience I think that the students we have entering our colleges have.

Linda: You are so right. You are so right. And we don’t do this to children anymore, okay? There is a good side to it—there really is—because you do buckle under and get very serious about your homework, very serious about your studies, or you look like an idiot the next day. Anyway, the sport I’m trying to think of is that sport where you get the ice or any kind of ice particles out of the way.…

Rebecca: Curling?

Linda: Curling! Curling, yes, we are curling teachers and curling parents. We try to clear the way for our students. We don’t want to put them in stressful situations. We don’t want to ask Johnny to read if Johnny might not be able to read—and I mean, get up and read well—and the problem with this is that students are denied the opportunity for achievement. And there is no achievement without the possibility of failure, there just isn’t. So students have no idea what fear and terror in school might be. There’s bullying and all that, but I mean from the learning experience. So no, they don’t have any kind of the experiences that I had.

Rebecca: How do we start coaching students then to become self-regulated learners if they’re coming out of this really different environment that’s much more supportive and doesn’t allow for failure seemingly?

Linda: They start failing in college. We’re still sort of, you know, curling them a little bit, but they are really facing a much greater challenge. They get insecure really quickly because they’ve been told how special they are and how smart they are, and then they begin to question that, because they’re not doing as well as they were in high school, where you could get an A relatively easily. Now, “Oh my, it can be really hard,” and then they start getting C’s, and then you have their attention. That’s a way that you can tell them that, “There are ways that you can get A’s, you did not learn how to study, here is a way to learn.” Does it involves a sort of effort? Sure, but it’s really just talking to yourself and deciding what strategies would be best for you, testing out strategies, seeing how they work, and you will be more successful. And there have been studies of students—like in developmental courses—that show that the students who are struggling the most tend to know the least about self-regulated learning strategies and start to do better if they use these strategies.Of course we’ve got to get them to use the strategies, we’ve got to explain these strategies. It can be life changing for them in the most positive way.

John: I think part of the issue is that faculty generally haven’t been taught these strategies themselves. They somehow found ways to be successful, so they become self-regulated learners, but faculty are the exceptions. They’re not the typical student, and they’ve never really been trained to teach students how to become more effective learners, in part because they never learned that directly themselves quite often. What can faculty do to be more effective in this way?

Linda: Well first of all, faculty have to realize that they’re the weird ones, and everybody else is normal. [LAUGHTER] So we have to stop projecting our learning abilities, our strategies, our interests in the life of the mind onto everybody else. We have to not only sell our material, but we have to equip students to learn our material. We don’t want to do that. We say, “They should know by now.” Well, guess what? They don’t. So what are you going to do about it? You’ve got to start from where they are. Teaching students learning strategies takes a couple of sentences every class. Now, if you really want to get into self-regulated learning activities and assignments with them, that might take a few minutes per class period, but you don’t have to do it every class period, and a lot of self-regulated learning activities can be homework, in which case they take no class time at all. This is so easy to do. This is why I think faculty have really been attracted to my book and why I’m asked to speak on it so often, because there are so many little things you can do that don’t take away from the content at all—rather, they reinforce the content—that make this huge difference in the performance of most students. You can’t always bring everybody along with you. There are some people—some students—who’d just as soon shoot themselves in their foot, but most do not. They find these activities so easy to do. They don’t take a lot of time and they get to know themselves and start doing better, so students don’t complain about this.

Rebecca: Can you describe what a couple of those activities might be?

Linda: Sure, absolutely. Well, let’s consider the different parts of the course and I’ll just give you just a few, some of my personal favorites. For starting a course for instance—starting and you can also end it with these sorts of activities—but one of them is a goal setting activity. You can assign this as homework, you can have students do it in class with students write on “How I earned an A in this course.” Now, you would be surprised and students will be surprised—C students and B students will be surprised—that they know what it takes to earn an A in a course, and they will come up with, “Well, I’ve got to come to class every day, don’t I? And in class I can’t fiddle around with my mobile device, and I have to start a paper sort of early and I have to keep up with the readings.” For many people, writing this down is goal setting for them. They think, “Well, you know, maybe I could do this, maybe this isn’t so absurd.” If you make a discussion out of it afterwards, the A students will say, “Yeah, I do these things, it’s not unrealistic.” And then the C and B and F students will say, “Well, let’s give it a whirl.” Then at the end of the course, you give them another little essay assignment, “How I earned an A in this course—or not.” [LAUGHTER] True confessions time, right? And so students assess how well they met their goals. Goal setting is definitely a part of self-regulated learning, the planning and then self-evaluation at the end. Another thing that you can do is you can give your students essay questions. If you give an essay final—or have any essays on it at all—you can give them the essays on the final to take the first day. This will not take much class time at all, because students will know very little, or they’ll try to BS an answer. So they will try but they can be really quite wrong. Now at the end, for the final, they correct their answers, and then rewrite these answers given the knowledge that they have gained throughout the course. This can be really interesting for faculty—for not just faculty… well, it can be interesting for them, too—because they can see exactly what they learned. So it is a measure of learning. Faculty will never get that comment on the student evaluations saying, “Well, I didn’t learn anything in this course.” Never again, that’s gone. So anyway, those are a couple of things that you can do. Little assignments you can make on the readings. Little reflection exercises like, “What did you think was the most important point in this reading? What surprised you the most? What connections can you make between what you read and your prior knowledge, what you already know? Or to your life? Or your emotional reactions to it, if the material is amenable to that?” So those are little reflections you can give on the readings. Another exercise, a self-testing exercise, is called “read, recall, review.” This is the best way to do reading. Forget about rereading, that’s what students really do… It’s really a waste of time. What students should do is to read a portion of the chapter or the whole chapter, put their notes away, close the book, and then recall as much as they can and write it down. Then they should go back and look in the chapter for what they forgot and what they might have gummed up. And they know that, “I didn’t really get that point.” And so they go back and look at it, and then they recall again. Read, recall, review. Studies that have been done on this showing it is so much more effective than rereading. It really doesn’t take that long, and then you actually have the material in your head, even in your long-term memory. You get retrieval practice, you get deliberate practice, so there’s nothing as good as testing yourself except—well, the nicest thing we can do for students is to test them. In lectures—I should say mini-lectures—it’s a good idea to have students do this. You stop, let’s say, every 15 minutes or so and have students do the same thing. Write down everything that they can recall, and then work with their neighbor to fill in the blanks—their own blanks—and ask any questions they have. First they ask their neighbor their questions, and then they ask you. This doesn’t take very long at all, maybe five minutes, but then you know that the students got it and can remember it. Again, most effective… studies done on this, too. So this makes students aware of their learning or their lack of learning. You can give students what are called active learning checks. You give your mini-lecture, and then you stop—and by the way, you can warn students you’re going to do this so they’re listening—and ask them, “Okay, what are the three major points in my last mini-lecture that I talked about in the last 15 minutes?” Then they write those things down—and it could be two things or four things depending—and turn them in. They don’t really have to turn them in, but you know, you might want to see them yourself. Then you reveal the three most important points, and they monitor and evaluate their learning skills. Now, students are motivated to want to learn how to listen to you, so they want to improve. According to a study that was done, they improve really quickly. The first time, 45 percent of students got all three points correct. By the third time, 75 percent of the students got these correct. Remarkable progress, really remarkable. Then there are meta-assignments. In a problem solving field like chemistry or math, we are denying students learning opportunities when all we do is mark the wrong answers as wrong or incomplete and then drop the subject. Students should be able to correct their mistakes to get half the points back, let’s say. In other words, they’re going to learn how to solve that problem if it’s the last thing they do. [LAUGHTER] Again, you give them some sort of an incentive, then they learn. There have been studies on this technique as well. It’s extremely effective. And students can learn not just from you, but in peer groups. Peers can help each other very effectively. There is a wrapper—they’re called “wrappers”—for an exam, a reflection that students do after they get their exams back where they answer questions like, “How did your expected grade compare with your actual grade? How do you feel about that?” So they have to look at the exam and your feedback. “How many hours did you study? Was that enough? What did you do while studying? Might you want to change your strategies? Why did you lose points? Were there any patterns that you see here? How are you going to study more effectively for the next exam?” This has been life changing for students because they’ve never thought about this before. They’ve never really looked at their exams, their mistakes. They drop them too, right? They don’t want to see what they did wrong. Yet these are the best learning opportunities possible, and they will remember them. We remember our mistakes, we learn from our mistakes, and it’s sad that we don’t stop and use those errors. These are just a small sample of self-regulated learning activities. I can give you many more. [LAUGHTER]

John: And there are many in the book, which we strongly recommend to people.

Linda: Yes, yes, yeah…

Rebecca: A lot of what you’re talking about seems tied to growth mindset as well.

Linda: Exactly, and this creates, this generates the growth mindset because students learn that they can learn, they can do better. Otherwise they feel like their learning is like the weather. “Maybe it’ll rain on me, and maybe not. [LAUGHTER] There’s really nothing I can do about it. Because it’s all about you, professor, you are responsible for my learning, just like the fates are responsible for the weather. [LAUGHTER] And if I’m not learning, you’re not a good instructor, or you’re pitching the material over my head, or your teaching strategies are wrong.” And so everybody else gets blamed, and then they start to realize, “Oh, I can do this.” Now, this isn’t the best news for them in the world because then they have to start taking responsibility for their learning. And that can be, for some students, a hard pill to swallow. For other students it will be very empowering, and what we want to encourage in students is that sense of empowerment.

John: And that’s especially important, I think, in freshman-level classes, because students generally don’t come in with that type of mindset. They’ve often been able to blame it on the teacher and do things over and over again until they get the grade they want or get the extensions and so forth with a focus on self-esteem in many classrooms.

Linda: Oh, yeah, self-esteem without achievement.

John: But it’s an adjustment. So if they come in with a fixed mindset, and they’re confronted with failure, it’s pretty easy to give up. So we need to encourage students, I think, to see failure as a learning opportunity as you’ve mentioned. As instructors, I think we have to somehow convince them of that, because they don’t come in naturally picking it up, but the techniques you’ve mentioned are very good for that.

Linda: You know, our whole society makes them feel they’re not responsible for their learning. Look at what happens in K-12. Students have to take standardized tests and if they don’t do well, who gets blamed and who suffers? The teacher and the school, and that’s nuts. In the final analysis, we teach ourselves. We are responsible for our own learning. Good teaching can make a big difference because we can be motivated or unmotivated by teaching. We can acquire learning strategies through teaching. So it’s not that students are just left adrift on their own, we do have to help them. We do have to put them in learning experiences where learning becomes attractive for them, or you can’t help but learn, right? But they’ve got to pick up that learning and run with it themselves.

Rebecca: So you mentioned the idea of encouraging students to see learning and the self-regulation as empowering. What about those students who are a little resistant to that because it’s surprising to them that they’re not getting it and they’re failing and that it’s going to be more work? What are some things that we can do to encourage those students to see things a little differently?

Linda: Yes, first of all, if they’re failing and they subconsciously want to—it happens, it really does —[LAUGHTER] there is not a whole lot you can do about it. They might need some counseling and they might need to get some help from professionals like psychologists. But again, it can be difficult for students to realize that the ball is in their court because it’s a whole different gestalt for them. The only cure for that is success—a little bit of success—where they start doing a little better, let’s say, on the quiz on the readings or they start being able to solve more problems. That’s really the only cure. And we are assuming that they want to be successful. Again, if they prefer failure, then they are responsible for their own failure.

Rebecca: Right, they’re the ones that are normal and we are not, right? [LAUGHTER]

Linda: Exactly.

Rebecca: Maybe that should be the refrain of this interview, right?

Linda: Yes. They are the normal ones and we are strange. And we always have been strange. We were the strange kids in school, too. [LAUGHTER]

John: In your book you mentioned the Dunning-Kruger effect as being a barrier to some students, that the students who don’t understand things as well often overestimate their understanding. How can we overcome that?

Linda: Self-regulated learning helps when we give them activities and assignments where they do self-evaluation, because the only way to learn self-evaluation is through practice, practice with feedback. And that feedback doesn’t even have to come from you; it can come from peers or a computer program. We don’t give students a lot of practice in self-evaluation, and they certainly haven’t had much of it in K-12l. But the nice thing is that when we have students look back to see if they met their goals, or to evaluate their study strategies, or to assess their mistakes and the reasons for their mistakes, it makes all the difference in the world. After low-stakes practice, you can introduce higher stakes self-evaluation assignments and see more savvy self-evaluations.

John: What recommendations do you have for faculty who’d like to start building more self-regulation? Are there small steps that faculty could take to get started on this path?

Linda: Absolutely. There’s a sense in which most of the assignments, most of the activities, are little things. Here’s a little thing you can start off the course with, I was talking about essay questions, but you can just have students do a little reflection the first day and then again on the last day about the subject matter, as in, “What do you think chemistry is? Why is it a science?” You can find out a lot about students’ misconceptions just by looking at these reflections. And then of course, they’ve hopefully corrected a lot of misconceptions by the end. This could take like all of five minutes the first day. There’s so many little things. Here are some ideas for experiential learning. It’s so easy for students not to make a connection between a simulation, an interesting role play, a service-learning experience, or field work to the course. So it’s important that whenever you do an experiential assignment or activity, students reflect on what they are learning—for a simulation, to look back and explain what their goals were, to evaluate how well they met their goals, to assess their strategies, to explain how their strategies changed and their responses to other players. It’s very important that students become conscious of what’s going on in their heads. Only by becoming conscious can they remember the strategies? [LAUGHTER] And then they can write them down and articulate them. You can have students do short papers associated with papers and projects where they record, while they are doing it, the process they are following. If you’ve given them a process to follow, they even have a skeletal outline of what they should be doing. This is a place also for self-evaluation. If you have students do a revision, oftentimes you give them feedback on what they should revise, and they may or may not read your advice. So you can have them paraphrase your feedback back to them and write out their goals for the revision. What are they going to do? What are their strategies for revision? These are just little things. Students start to realize the value of this. And again, this is an assignment where you can’t screw up. It’s not a test, it’s just a reflection of what’s going on in your head. Students like to learn about themselves. And this is like the reading reflections. This is no stress. How do you mess this up? It takes less stress to just write an honest answer than it is to make one up that sounds credible. [LAUGHTER] I want to make faculty aware that the activities don’t have to be graded at all. The assignments don’t really have to be graded. You “grade” them pass-fail. Students pass just by completing the assignment. Let’s say, you had them answering three questions, three reflections. Did they answer three reflections? Is it vaguely on the chapter? Okay it’s not about football, it’s something about what’s in the chapter. [LAUGHTER] And did they meet the length requirement? It’s always a good idea to give length requirements on these reflections because for students, length means depth. So if you ask them to write a minimum of 150 words, you know, they’ll tend to do that. Those who don’t fail.You don’t count every word that students write. You eyeball the reflection. Essentially you are “grading” pass-fail at a glance. It doesn’t take much time. Plus, it gets students to do the readings in the easiest way and most productive way for them. It’s all about them, and it’s not about us. We just have to hold them accountable in some very quick way. Even the longer assignments that you might associate with a paper or project can be graded pass-fail. You have to make them worth some points if you’re still on a point system. But there are alternatives, that’s what specifications grading is all about. You don’t have to use points. In any case, you do have to at least eyeball the reflections and give some value in your course however you are grading. That communicates to students that this is important to you, that you put value on this meta-assignment or assignment wrapper, as you might call it. The same thing with the post-exam wrapper, these reflections on this exam. You make students do it because it’s worth 10 points if they simply complete it and hand it in—even though it’s for them—and they will realize right away that there is some value to this. Again, for some students, it will be life changing in the most positive way. And they will start to realize the way that they’ve been preparing for or taking exams may not be the best. They will realize what they tend to do when they’re taking an exam, such as to misread the question, or to be careless, or to not budget their time, or to not really thoroughly study all the material. Cramming is not very effective. You don’t have to spend time grading these exercises or giving any feedback at all. They can give themselves their own feedback. If they did it and they get the 10 points, okay, that’s plenty of feedback for them. They did it. You regard it as their meeting the requirements of the assignment.

John: And this is a topic you cover in another book on specifications grading, which is also another book we’d like to recommend. We’ll include a link to both of those on the show notes.

Linda: The title of the book is Specifications Grading: Restoring Rigor, Motivating Students—it’s been found to be motivating—and Saving Faculty Time, saving you time. If there’s one thing we don’t have, it’s time. Time is really more precious to us than money. Otherwise, we’d be working in some venture capital firm or something. [LAUGHTER] But time is really quite a precious thing for us. So, in terms of these sort of assignments, for self-regulated learning assignments, they’re all what we’d call “specs graded.” You set out the specs, they’re very simple, and you just grade them pass-fail.

Rebecca: I think pointing out how it doesn’t have to be complicated for faculty is important because I think we all want students to learn. We all want them to be self-regulated learners.

John: We all want to give students feedback, but we don’t want to make it impossible for us to keep up with our work.

Rebecca: Yeah. Or feedback that’s going to get ignored anyways.

John: Right.

Linda: You’re worried about students reading the feedback, and our feedback is valuable. We’ve given it, we’ve taken the time, and so you make them paraphrase it back to you. And this could be a learning experience for us because we might be “misread.” Students might not understand something that we’ve said. Awkward—that’s my favorite one—a sentence structure is awkward. What does that mean? That student didn’t set out to write an awkward sentence. That in itself will not help them because they don’t know what you’re talking about, and this is most unfortunate. But again, it’s a learning experience for us and we can learn to express ourselves somewhat differently. Too often, students get back a paper from us and look at the grade, read the paragraph at the end of the paper, and put it in their “circular file.” They dump it. They don’t read that feedback, so how are they going to get better? So paraphrasing our feedback back to us can be a very valuable exercise for them. And you can let them gain back some points for it. I just think that faculty should look at themselves as responsible for helping our students learn. They don’t come to us with those skills. We can be the finest instructor in the world, have the most interesting classes, hold their attention, and motivate them, but if they don’t know how to process that material in their own minds, it’s all for naught. Now, maybe, hopefully, seniors have learned to learn their material along the line. And by the way, there can be different learning strategies for different subject matter. There are different self-regulated learning activities and assignments for problem solving mathematically-based fields, and different ones for the social sciences and humanities. There can different kinds of assignments, different kinds of readings, actually different kinds of lectures. So we have to respect that. But we have to become conscious of study strategies, learning strategies, our strategies, and other strategies that are out there. But self-regulated learning strategies, to my mind, they’re the shortest distance between two points. Shortest distance between ignorance and learning because it’s all going on in your head, and it’s so powerful. The value of it to students becomes evident rather quickly.

Rebecca: And it’s a skill that can help through a whole lifetime, not just while they’re in college and I think helping students realize that is also really valuable.

Linda: Like no other generation before, these younger generations are going to have to learn to learn on their own. They’re going to have to keep up with their field, whatever their field is, and they might have to—will very likely have to—pivot into another field because their first field might run its course. They’re going to have to learn on their own. They aren’t going to have employers holding their hand. Not at all. They’re probably going to have to learn online, where you really are responsible for your own processing, more so than you might feel let’s say in a face-to-face class, and for your own motivating as well. There needs to be more motivation than simple fear that you will go hungry and won’t be able to get a job. [LAUGHTER] Yes, students are going to really have to learn how to learn. If they consider that a bitter pill, that’s too bad. This is reality, this is life, and most of them have not learned that life is hard. Many of them are wondering where their next meal is coming from, but a lot of students have not. Students need to learn along the line that life is not easy, that nobody does curling on their path. And they will face challenges, but if they have the strategies for facing these challenges, no problem. They needn’t be paralyzed. They needn’t freeze.

John: You foster some really good advice and I think our listeners will appreciate this and it’s really powerful.

Rebecca: Yeah, I agree. We’re all wondering, what’s next for you?

Linda: I’m actually Director Emeritus. I’m actually retired from Clemson University. But you know how academics are, [LAUGHTER] they don’t disappear, they just sort of like fade away. So I’m trying to ease into retirement because it’s not an easy thing to do? Not when you love what you’ve been doing. But I have sworn off writing books. That’s progress! [LAUGHTER] I’ve written some articles and chapters in other people’s books, so that’s fine. And I’m still traveling to give keynotes and faculty workshops. That’s hard to give up because it’s interesting to go somewhere else, somewhere new.” And I still give webinars and podcasts. But eventually I won’t be doing that anymore. Ultimately I want to work with animals. I do love animals but I’m still busy doing this and still loving doing this, but also loving just as much not having to do bureaucratic tasks for the university [LAUGHTER] and not having to stay up until two in the morning doing my email. When I’m traveling, not have to worry about, what’s going on back at the office. So I’m not complaining about retirement. I really like where I’m at right now, but I know that I will eventually fade into the sunset. That’s okay because then I’ll reinvent myself.

Rebecca: Sounds like some self-regulation was going on there, I’m pretty sure. [LAUGHTER]

Linda: Yes, I’m trying, I’m trying to retire but not too quickly.

John: Well, we’re glad you haven’t, fully yet.

Rebecca: This was really great. Thank you so much for spending some time with us today.

Linda: Well thank you for this opportunity. I hope that I have helped some faculty members out there to help them help their students to achieve more, because again, we all do want our students to learn. We’re all in love with our material, it’s worth learning, and we just have to help our students do that. So thank you ever so much, and thank all of you listeners for listening.

Rebecca: Thank you so much.

[Music]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Kim Fisher, Chris Wallace, Kelly Knight, Joseph Bandru, Jacob Alverson, Brittany Jones, and Gabriella Perez.