293. Study Like a Champ

The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, Regan Gurung and John Dunlosky join us to discuss a new resource they have created that is designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

Show Notes

  • Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. Routledge.
  • Gurung, Regan and J. Dunlosky (2023). Study Like a Champ: The Psychology-Based Guide to “Grade A” Study Habits. American Psychological Association.
  • Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological science in the public interest, 9(3), 105-119.

Transcript

John: The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, we discuss a new resource designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Regan Gurung and John Dunlosky. Regan is a social psychologist and is an associate vice provost and Executive Director of the Center for Teaching and Learning at Oregon State University, He is the author of over 120 peer-reviewed articles and has co-authored or co-edited 15 books. John is a professor in the Department of Psychological Sciences and the Director of the Science of Learning and Education Center at Kent State University. He co-authored the first textbook on metacognition and has edited several volumes on education. Regan and John are the co-authors of Study Like a Champ, which was published earlier this year by the American Psychological Association. Welcome, John and welcome back, Regan!

Regan: Thank you.

John D.: It’s great to be here.

Rebecca: Today’s teas are:… Regan, are you drinking tea?

Regan: Just water because it is a scorcher here in Oregon. So water, and lots of it.

Rebecca: Hydration is important. How about you, John?

John D.: No tea for me for about five years, I used to really be addicted to oolong. And now I’ve gone full espresso. [LAUGHTER] Water for me today too. It’s a little bit late for espresso on the East Coast.

John: And I have an Irish Breakfast tea today.

Rebecca: Oh, John, you’re really pumping up the caffeine today. Also on the East Coast,

John: My day started at 5:30 this morning…

Rebecca: So did mine.

John: …and it’s going to be going really late. We’re in the midst of grading here, we’re recording this a little bit earlier than it will be released. So we’ve invited you here today to discuss Study like a Champ. What prompted you to create this book?

Regan: So both of us have been teaching for a long time, both of us do research on teaching and learning. And I think both of us really like taking stuff from the lab and testing it in the classroom. And, I think, as passionate teachers, we noticed that we knew the stuff, we knew what students should be doing, we told our students what they should be doing. We even tried to design our classes in that way. But clearly, we needed to do more of it. And the impetus behind this book was let’s put these expert tips in the hands of students everywhere. We didn’t want it to be just those students who had teachers like us who talked about studying, we want it to be in student hands. And that’s where it came from, specifically driven by the fact that even though there’s some really neat stuff out there on learning, very little of that, if any, is written to the student. And that’s something right from the get go when John and I first talked, we agreed that this would be in a voice that would speak directly to students.

John D.: Absolutely. In fact, I think that was the main impetus because of all the wonderful books and volumes out there on learning sciences. And just too many of them, I think, appropriately so written for teachers. And often they tend to be a little bit fact listing because of that. In other words, I’m going to tell teachers everything there is to know about learning, which can be overwhelming. And what Regan and I wanted to do is kind of find the most effective and best little snapshots of the learning sciences to share with students, the things that we think will move the needle the quickest, so to speak. So by no means do we tell students everything there is to know about the learning sciences, but we hope we tell them the best stories to get you back onto the learning track, so to speak.

Rebecca: So you begin your book by discussing a variety of widely believed learning myths. Can you talk about a couple of these myths?

John D.: Sure, one of the biggest and I really don’t think this holds true for college students, but K through 12, definitely, it’s just the myth of learning styles. So, we all have learning preferences. I certainly have preferences. I wish my instructors would sing to me all the time and do my learning through music. It’s just that’s not going to help me learn calculus, and so forth, where really hard work needs to be done. So it’s natural to accept a myth like that, because it makes everyone feel good. It really can also undermine learning because it makes us want to do things that are more aligned with our preferences, but sometimes misaligned with most effective practices. So that’s kind of one learning myth, which I hope, in 10 years, if we were to have this discussion again, it would not come up because it was completely dispelled. But there are a variety of others too, that I think can undermine student success too.

Regan: And I think something else, that we really tried to put ourselves in students’ shoes, and we built on what our students said to us. And another one I wanted to highlight in response to that question is so often I think our students think that learning is up to us, if they have a good teacher they learn. If they have a bad teacher, they won’t learn. And undoubtedly, teacher effectiveness is important, but it’s also up to the student. The student has to do something, the student has to take accountability. And early on in chapter one, we show that at least half of the variance half of the accountability for learning is in the student hands. Teachers are also responsible, but really, if a student isn’t doing their bit, then learning is not going to happen. And now once you are ready to do your bit, well the rest of the book tells you what to do. But you’ve got to realize that don’t just sit back and go “Come on teach. Do your magic.” We all like doing our magic. But we can do all the magic in the world if you are not joining us in this adventure, and I’m thinking adventure a lot, because Zelda Stairs of the Kingdom just got released recently, and my household seems to be quite into it. So yeah, join us on the adventure.

John D.: Absolutely. Let me just jump in with one other learning myth that really gets me, and it literally gets me, because I fall prey to the same thing. That’s if you’re struggling, you’re not learning because it’s frustrating. And I often try to pick up new skills, I’m teaching myself to play the guitar now. And I struggle, and I get frustrated, I want to throw my guitar down because things aren’t happening. And I have to remind myself that sometimes it’s the struggling when you’re doing the most learning. And if we could allow students to realize that if you’re not struggling, you actually may not be learning, and that you should embrace the challenges, the hurdles that are in front of you. And it turns out, some of those hurdles are embracing the most effective learning techniques, which produce struggles in learning, but yet also produce the best long-term outcomes. Whereas other ways to prepare for classes and so forth, the way I’d like to teach myself to play guitar, makes it seem very easy. But then there’s really no progress being made. And if there’s any method I could really want to undermine, that would be a major one, that sometimes struggling is a good thing, not all the time, and it’s up to the students and the instructors to figure out is this a good struggle or bad struggle and how to correct that.

John: Those times when we try to give students desirable difficulties, they often find those to be somewhat undesirable. And I often get comments in my course evaluations that “he’s making us learn it ourselves.” And my response is always “Well, I can’t really learn it for you, I can give you the tools, I can give you resources, I can help you learn this material, but ultimately, you have to do the learning.” Going back just a little bit to your comment, John, about learning styles, I had a discussion in an online class, which is mostly upper-level students. And about three quarters of the students in my class were very strongly convinced of that myth of learning styles. So it hasn’t gone away. We hope it ma, but it is very pervasive, certainly in K through 12, where they’re regularly testing it, but some of the students have mentioned that they’ve been tested here at the college in some of their classes. And it’s again, really frustrating to see that, but it’s pretty pervasive.

Rebecca: Yeah, I was struck too, John, when you were talking about both learning styles and the struggle, because as I was working with mostly graduate students this semester, I had those exact conversations with those students. But it may have been the first time that they’ve really come up with some barriers that they faced in learning, and really had to reassess how they were working towards their goals.

John D.: Absolutely, and nothing but utmost compassion for students and sympathy, because in many ways, the experiences they had before college allowed them to use ineffective strategies and still get by. So things didn’t seem like a struggle, the difficulty now, the first time, then you hit a struggle, when you really have to change strategies or techniques, you may not realize what to do. It might seem frustrating and overwhelming, when in fact, now’s the time when the real learning begins.

Rebecca: Yeah, and I think sometimes when the subject matters new, as well as the strategies are new, students can immediately feel a little cognitive overload. [LAUGHTER]

JOHN D. Absolutely.

John: And part of that I think may be related to how they’re often taught throughout their whole career. In general, much of the advice they get in elementary and secondary schools, and sometimes in college, encourages them to adopt strategies that may not be that effective. Often, people still use high-stakes exams. And people don’t do a lot of interleaved practice or spaced practice, which are things you talk about really nicely in your book. And the incentive structure for students is perhaps to favor things that align well with short-term recall without that longer term learning. And your book can help, if we can get students doing this, but it seems to me like we also have to do a bit more work and getting faculty to adopt some techniques that may encourage students to use techniques that are more effective. Might this be something we could build into our classes a little bit?

Regan: Actually the riff over a couple of different things you said, John, I think there are two things going on here. One is, a lot of the issues that we run into with our students are actually pretty easy to pinpoint. And that is high school. And when I say high school, I’m not saying “Oh, high school teachers,” I’m saying the whole high school environment. One of my favorite examples is the syllabus. Many of us college and university faculty complain that students don’t read the syllabus, and I had a great conversation with my 17 year old. I was talking about the syllabus and he said, “Yeah, the syllabus is useless.” I said, “Excuse me?” and he said, “Here’s what we do with the syllabus. We spend a whole day, a whole day, reading through every word on the syllabus, but the syllabus is not really designed to help us learn.” And I looked at his syllabus. And sure enough, the syllabi he had didn’t really map on to the best practices that many of us listening employ. And I could completely see why he had that whole. “Yeah, I’m not going to read the syllabus when I get to college. I said, “And of course, you will now,” but it was a great conversation where I really got to see the mindset. And I think that’s just one example, where a lot of this happened, where it’s like, let’s see, what have they just been used to? Then add on top of that, how learning changed during remote learning. That’s another whole mix to the whole issue. And then, add one more, the misinterpretation of, I think, learning styles. Before we started this call, we were joking about the fact that our book is now in an audio version. Now, because it’s an audio version, does it mean the auditory learners will be better than those….? No, it just means that sometimes when you want to listen to something that works well, other times reading something works better. So it’s not this whole “Oh, look, there’s this one modality and I’m a person of this one modality.” And I think that preference that John D. so nicely used… it’s preference, right? …and we have preferences. But let’s also not forget the take home message of that Pashler et. al 2008 study, which is the more styles that you use…. And here’s where it goes back to desirable difficulties, mixing it up, is actually better for learning, no matter what your preference may be.

Rebecca: Oh, but mixing it up is so uncomfortable. [LAUGHTER]

John D.: It makes things a lot more difficult, right? One advice I would give too, that my teaching has changed dramatically, as I’ve gotten more into learning sciences. And at least for someone who teaches, I hate to say this, a non-essential course course that I love and courses that I love for undergrads, I realized I was teaching them too much. So I put them into almost a defensive mode, where the only way to prepare, because they were overwhelmed, is to cram as much as they could and hope to pass the exam because it was a tsunami of information. So over the years, I’ve followed the model, less is more, it’s up to me as the instructor to figure out what is really essential, and how to give them just as little as possible to make their experience as large as possible. So that they then can use the techniques to learn the content well enough so that they can keep that content with them a lot longer period of time. So it’s really kind of changing styles. Now, don’t get me wrong, if I’m teaching chemistry to pre meds, and you need to know chemistry, then you need a different approach, because there’s a lot to learn, it’s difficult, and aspects of our book really focus on how to meet those challenges too. And I think students, just like instructors, need to pick and choose where they want their battles, you don’t have to ace everything as a student, but decide what you really want to do well at, and then use the most effective practices to nail it as you work forward toward a long career and lifelong learning.

John: One of the things you include early in your book is a list of what research finds to be effective and what research finds to be ineffective, side by side, which for faculty use could be really helpful except for some degree of similarity between those two lists. Could you just talk a little bit about that.

John D.: I like to think about it, as much as I’ve not talked about it like this, as more or less effective versus effective and ineffective. And I’m not gonna like reference to that table necessarily, but let me talk about one that I badmouth a lot, which is highlighting as a learning strategy. And like Sharpie has not contacted me with a lawsuit or something because it sounds like I don’t want people to have highlighters. And what we mean by less effective as a learning strategy here is, is that using a highlighter is just the beginning of a learning adventure and not the end of it. So there are great uses of a highlighter of rereading material, of these things that really don’t lead to a great deal of learning, because they kind of are the stepping off point or the catapult for learning, so to speak. So is using a highlighter ineffective for learning? Absolutely, it doesn’t help you learn the content that well. But what I recommend to students: to highlight everything they want to learn as kind of a different approach to that strategy, so that they then apply the most effective strategies in learning it. Yes. So as much as we kind of play off ineffective to effective, it’s more like every strategy in its right place. And some strategies that can be generally ineffective for very specific uses might be relatively good, and what I think Regan and I try to do, because it’s way too much to think about if you’re not a learning scientist, like one exactly show us the specialty strategy, but we try to focus on are just strategies that students could learn just more generally to do well in any particular content. So kind of low hanging fruit, so to speak, where you don’t have to be a learning scientist to make decisions about which specialty strategy to use in every place. But here’s some kind of fail-proof strategies that can help you anywhere.

Regan: I want to actually say there’s that flipside too and John, this is something you asked about earlier when you talked about teachers changing what they do, and I think John D, and I joke about the companion volume to this is Teach Like a Champion. But I think the nice thing about that table of high utility and low utility strategies is that it’s not just the student who needs to be aware of the fact that some are high and low. But I think we take it further to say why are some of those high and low, and then here’s the key companion piece, which is, we hope that instructors reading this can also take those tips, and be ready to share different options with the students. Because I know both John and I have compared notes on this, when a student comes to our office after an exam who’s not thrilled with their score, who has not scored well, we both do some pretty intensive. “Alright, so tell me exactly what you do.” Just that conversation is so important. We both ask to see their notes. And of course, every once in a while a student says I don’t take notes. Well, let’s start right there. And that’s why we have a whole chapter on note taking, and why take notes and how to take notes. But I think that’s exactly it, is so many of us. and I’ll say us, not just our students, so many of us are so used to doing things just one way that we’re not ready. And we don’t look enough about is this really working or not. And I think that’s what we really push with a lot of very real examples ripped from the headlines of our lives, as it were, where we talk about, look, this is what somebody’s tried, and here’s what they did wrong. And here’s how they can make it better.

Rebecca: I wanted to talk a little bit about that chapter on notetaking. Faculty often make the observation that students don’t take effective notes, but then don’t necessarily respond by teaching students how to take notes or giving them some strategies to improve that skill. Can you talk a little bit about some of the things that finally might show students or offer students to take more effective notes?

John D.: Absolutely. So the one thing that I would recommend highly is that if we all agree that taking notes in its own right is not where the learning occurs. And in fact, research comparing people who take notes versus who don’t take notes typically showed no differences between the two. Now, the key is, when you look at these studies carefully, you’ll realize the folks who took the notes, and who didn’t take notes do equally poorly. So it’s not like they do equally well. So the notes are a artifact that in many cases students can use to really learn the content well. So the question is, how do you get all the correct and appropriate information into those notes? So one thing that teachers can do, that research shows really helps well, it’s just flag important content, literally tell students “Okay, time to take notes, this is critical, I need you to know this.” One of my favorite statements, ”The next thing I say, will be on the exam.” [LAUGHTER] And see how many people start taking notes. Now, of course, I would love to see notes that are embellished with examples, with questions, and so forth. But at least getting the rudimentary content down there as an artifact, so students have it to utilize as they’re preparing is so important, or having teams of students take notes and share those things as well is also good. So I think for teachers to be compassionate, but also to help your students sometimes identify what’s most important, by speaking louder by saying, “Hey, this is really critical.” Sometimes I have to even flag it to not only just critical, but it’s going to be an exam, you’d think those two things meant the same thing, but sometimes students need a little help. I think it’s important, and there’s nothing wrong with telegraphing what is most important in your class. In fact, let me restate that. There’s everything not right about not telegraphing. Now, try to parse that. But what I’m saying is, we should be telling the students what they need to learn, how we’re going to test them on it, what we’re going to test on, and allow them to meet those successes. And I think good note taking, and helping students take notes by telegraphing what should be in there, can be very useful, in my own experience, and at least some of the research too.

Regan: Notice what else is in there. If we do want students to take good notes, we don’t want to be talking extremely fast. We want to make sure we have pauses, we want to make sure our slides, to the extent we use them, are not so packed with information that either they want to copy it all down or they get it all as a handout so they don’t have to, but there are all these nuances. And I want to go right back to that finding about the students who take notes and the students who don’t take notes. The bigger issue is what are you doing with those notes? I think far often students just think about it as “I’m going to record what the instructor said.” Well, you’ve got to go back and revise them. You’ve got to go back and check and see if your understanding is right. You need to review them. The issue is, I think, when people take notes, they’re not doing enough with those notes. And there are ways where we can talk about how to take better notes and they’re not. When I teach 100, or intro level classes, every once in a while, I will stop, and I do this during the first week, I will stop. And I’ll say, “Alright, here’s a reality check. If you took good notes, you should have the following. How many of you have it in your notes?” And I do that a couple of different times that first week or the first two weeks to get into that habit of you should be taking notes, you should be doing something about it. I also want to say something that didn’t make it into that chapter on note taking, but it’s gone up another shot, because I’ve seen this so often, in the last year and a half or two years, students taking photographs of the slides. Reality check, people, that’s not taking notes, taking a photograph of a slide is not taking notes. But I think in some students’ minds they’ve taken notes, just more effectively. And there are studies now rolling out that show that the photograph slides versus note taking versus controlled no notes. that the photographs and no notetaking is doing the worst on quizzes. But I think that’s something else we’ve got to take up” how you take notes is important and taking a photograph is not it.

John D.: Let me spin back to something that Regan said that I think is great, too, as far as helping students develop better notes, I begin, and I know Regan does this as well, every class with a no-stakes quiz. And I use that for a variety of different reasons, some of which we talk about in our book, but one of them is first try to answer the quiz multiple choice question without the alternatives from memory. And if you got it from memory, that’s important, because that’s going to help you learn. And then if you don’t get it from memory, now see if you can answer using your notes from the last class. And if they can’t answer the question using their notes, we have issues. So they’re not taking complete notes, they don’t understand their notes, or the issue could be with me. Occasionally, I look at the notes and realize, “Oh, my goodness, I must have said something wrong last class, because everybody’s notes are incorrect.” [LAUGHTER] And if everyone’s notes are incorrect, the source of the error is probably me. So it’s a way to help students understand that the notes are the vehicle to understand the content that we really want them to learn. And there’s ways to reinforce that through no-stakes quizzes, and so forth to help them understand that what we’re doing in the classroom is providing all the scaffolding to help them succeed.

John: One of the things you emphasize early in the book, and throughout the book is the importance of students developing their metacognition as well as their self-regulation skills. Could you talk a little bit about why that’s important, because I think that’s something that’s not emphasized enough throughout their educational experience,

John D.: I’m gonna just focus on one aspect of our particular viewpoint on self regulation, and then I’ll let Regan go with the more fun stuff. I want students to really succeed in school and in college. But quite frankly, as a college educator, I want students to succeed in life after they leave here. And one of the most important skills that anyone can learn is time management, that’s about self regulation. And believe it or not, I have actually students tell me, it’s like, “Well, I know, managing my time and developing plans, that’s just not my cup of tea.” (speaking of teaching for tea). But it absolutely should be, because effective people develop goals, plan on how to get them, and manage their time to get there. So if students can take something from this book that I think is highly valuable about self regulation is to look at our examples and our encouragement and inspiration to manage your time, make goals and plans, and develop a time management strategy that works well for you. Because that’s ultimately going to lay the foundations for a successful use of other strategies we discuss in the book, but also for a highly successful life, whether it be success in your job, your hobbies, raising a family, or whatever, successful people manage their time and self regulate. That’s something that we really do push in the book, and some of it’s not necessarily metacognition, per se, but it’s about really taking ownership of your life and deciding how to get things done effectively. So time management, for me is so important aspect of this book.

Regan: I think whenever you think about self regulation, I really try to stress two elements of it on one hand, especially think about metacognition is the: “What do you know? Do you know this stuff?” And there’s that whole “Am I ready for this exam? Do I really understand this concept? Do I know how to do this?” That’s the classic metacognition, but even more broadly, when we talk about plan and monitor and assess your knowledge, that’s, I think, really important as well. And of course, the tips such as retrieval practice is great for the assessment too and to assess yourself. But I think I love the way John talks about time management because when we talk about self regulation, a big issue that I hear from students a lot is where those two things interact, not being able to regulate how they use their time, especially when it comes to social media. And students will say to me, “Yep, I start scrolling Tik Tok, and before I know it’s an hour later. Well, that’s a basic self-regulation issue. That’s a time management issue. That’s a planning issue. And I think in our planning chapter, we recognize this and we tell students “Look, it’s okay to enjoy social media. But guess what? Plan your social media timr. So it’s not like you’re saying, I’m not going to use social media. Be real. If that’s what you enjoy, sure, look at some social media, but plan it out, allocate time for your social media, just like you allocate time for your work so that everything fits into your schedule.”

Rebecca: Allocating time, it feels like have a lot of conversations with students about that on a regular basis.

John D.: Oh yeah.

John: Are there other topics you’d like to emphasize about your book that we haven’t touched on yet?

Regan: Well, I think one thing that if you’ll notice, the last chapter in our book is all about things that, for the longest time, nobody thought related to studying: sleeping, eating, physical activity. But I think that is just so important. That is so important. And gosh, I know, I don’t think I ever shared this with John. But sometimes I almost played with putting that chapter first. We did put it at the end, because I think it’s really important. But I think most students don’t realize how those life things interact. And I think for those of us who read up on higher education, there’s just so much written about burnout among students, among faculty, and all of that relates to those things we talk about in the last chapter. It’s prioritizing sleep and prioritizing good eating and prioritizing physical activity, prioritizing just getting out and getting some air. And some of my favorite comments that I get back from students… it happens a lot when I teach health psych… but even otherwise, where they will say, “you know, yeah, I caught myself scrolling, I went outside for a walk, and it was just five minutes, but I came back and I could tackle things better.” And I think, people, it’s 360 degrees of living is what we’ve got to practice.

John D.: And I love that Regan. Because obviously, we saved the best for last with that chapter. But, come on, to do all these things. It’s a time-management issue. If I’m going to work, all of this in, how can I do so so I have enough time for play and I have enough time for work, which are two important aspects of life. So, totally agree.

Rebecca: Well, they always wrap up by asking: what’s next? [LAUGHTER]

John D.: Wow, that’s a big question. And what’s next, I would say for me, the strategies that I think are most effective, that work the best, unfortunately, have also been investigated the least, because they’re difficult to investigate, they involve multiple sessions for students to engage the same material, and so forth. So as someone who was kind of born and raised in a laboratory, either my own lab or laboratory of the classroom, what I’d love to see is just much more evidence-based research focused on further understanding, not only what works best, but how students can engage in the best practice in the most efficient way. So it’s balancing both an understanding of wanting students to obtain their learning objectives, but also understanding that they want to do so effectively and efficiently. Because there’s so many aspects to life beyond just school. So I’m always going to say more research, although I could totally understand some would say more application too, but I’m gonna go what’s next is some more research and all these really effective techniques and how to use them more effectively.

Regan: And it’s that last part, John, that is what’s exciting me right now, I’ve been learning a lot more about the whole field of implementation science, which is just because something is effective, and something is efficacious, what are the factors that influence its implementation? …because whether it’s a student reading our book and trying to implement our recommendations, or a faculty member, instructor, teacher listening to us and trying to implement what we’re saying, there are still nuances, there are still contextual factors. And I think we’re just getting better at intentionally and systematically teasing apart what some of those issues with implementation are. And for me, that’s been pretty exciting. I was fortunate to read a lot of a fellow Oregon State University, new PhD, Dr. Rachel Schweitzer, who’s done a lot of this work on implementation science, and some of her ideas. I’d love to be able to test them in my class, to dovetail with continuing research on study techniques and how to make it happen more,

Rebecca: Some really important stuff that needs to happen, for sure.

John: And thank you for this book and all the other research and work that you’ve done in support of improving student learning.

John D.: Thanks for having us on.

Regan: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

290. Transparency in Learning and Teaching

While instructors know what they expect from students, these expectations are not always clear to their students. In this episode, Mary-Ann Winkelmes joins us to discuss what happens when instructors make their expectations transparent to their students.  Mary-Ann has served in leadership roles at campus teaching centers at Harvard, the University of Chicago, the University of Illinois, the University of Nevada – Las Vegas, and Brandeis University and is the Founder and Director of TILTHigherEd.

Transcript

John:While instructors know what they expect from students, these expectations are not always clear to their students. In this episode, we explore what happens when instructors make their expectations transparent to their students.

[MUSIC]

John:Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John:…and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Mary-Ann Winkelmes. She has served in leadership roles at campus teaching centers at Harvard, the University of Chicago, the University of Illinois, the University of Nevada – Las Vegas, and Brandeis University and is the Founder and Director of TILTHigherEd. TILT is an acronym for Transparency in Learning and Teaching in Higher Ed. We are very much fans of the TILT approach and have referred to it often in workshops on our campus (and on previous podcast episodes). Welcome, Mary-Ann.

Mary-Ann: Thank you. I’m really delighted to be here with you. And I appreciate the opportunity to speak with you on Tea for Teaching.

John:We’re very happy to have you here. You’ve long been on the list of people we’ve wanted to invite. So we’re very pleased that you’re here today. Today’s teas are:… Mary-Ann, are you drinking tea?

Mary-Ann: I am indeed. And I’m drinking a Sencha green tea today. That’s my new favorite kind of green tea, Sencha.

Rebecca: Nice. I have English breakfast today.

John:And I am drinking a mixed berry Twinings black tea…

Rebecca: Hmmm.

John:…which I haven’t had in a long time. I wanted to mix it up a little bit today.

Rebecca: …mixing it up with mixed berries. So, Mary-Ann, can you tell us a little bit about how the TILT project came about?

Mary-Ann: Sure. This was years back, I want to say in the early 2000s, late 1990s, where I was working at the BOK Center for Teaching and Learning at Harvard University. And I was leading a seminar group discussions about teaching and learning. And we began to think about the question: “What happens when you tell students why you’re teaching how you’re teaching, just what happens when you tell the students more about your choices as an instructor, how you’re choosing to shape the learning experiences for the students?” And that’s not often something that we think about first when we’re thinking about what’s the content of the course. But we began to think about that a lot. And we had a kind of metaphor about the Wizard of Oz, and pulling back the curtain to show what was happening behind the scenes to build the experience. And then somehow through that conversation, the word transparency emerged. And that became the word that we used pretty regularly from that time on. When I moved to the University of Chicago, that was the word we were using, and it kind of stuck. So that’s kind of where it started. And it started alongside of my career as an educational developer. And it’s kind of been, for me, in the background or on the side, as something that I’ve been kind of tracking along with as a project. It’s still there, it keeps going. And just about a year ago, I began to work on TILT as my full-time job, which I’m really happy to be doing now because it gives me an opportunity, not just to do a guest talk here or there, or a keynote address, which is usually a one time-interaction. But now I have the flexibility to connect with institutions around a longer-term project. So if there’s a faculty learning community that emerges from a first talk that I would give, I get to follow up with them later and see what’s happening and check in with them. Sometimes I get to see the assignments before and after, which I really like. And I invite those now, because we’d like to publish some of those on the TILThighered.com website. And there are some schools that I’ve been working with in the state of Washington for several years now running with their TILT projects. And that emerged from a project we did with the entire state system of Community and Technical Colleges in Washington State. So I have opportunities now like that, where I can work with larger scale TILT projects that take more time, because this is my full-time job now. And I’m really happy about how that’s working, because I feel like it’s getting larger beneficial impact for students in a way that’s more efficient than when my full-time job was at an individual institution.

John:Could you give us an overview of the TILT framework?

Mary-Ann: Absolutely. So the TILT framework is meant to be a very simple tool that is a framework for an ongoing kind of communication among teachers and students. And in all of our studies, we asked teachers to use this framework in their own way at their own discretion, because we know that it’s not really possible to expect that people would do the exact same things with it. So our research is based on the premise that people are using this framework in their own way, at their own discretion, in a way that feels consistent with their teaching style. So there are three parts to this framework: purpose, task, and criteria. And what we ask in all of our studies is for teachers to engage students in conversation about three aspects of a particular assignment or a project or even an in-class activity. Before the students do a piece of work that we want them to complete, we’re asking for teachers and students to have a conversation about three aspects of the work before the students start working on it. And those three aspects are the purpose, the task and the criteria. Now the purpose kind of consists of two pieces. The first part is talking about the skills that students will practice while they’re working on the assignment. And then how are those skills useful, not just now in this course, or maybe in college and other courses, but how are these lifelong learning skills that will be useful for the student in their careers after college or in their lives ongoing? And then the second part of the purpose is about the content knowledge. What new information or what disciplinary information will the students be researching, or gaining, or applying when they’re working on the assignment? And how will that be also similarly useful to them, not just now, or in college, but beyond in their lives? The task, that’s the second part of the TILT framework, and the task is sort of about what are the teacher’s expectations about how students will approach the work? And for the students, it’s kind of like mapping out their game plan, like, what’s the first thing they will do? Will they Google something? Will they go to office hours? Will they go seek out a research librarian? Will they go into the lab and start mixing something like, what’s the first thing they’ll do? And then a sequence of what they plan to do after that until they submit the work. In an ideal world, the teachers and the students would have similar expectations about how that would go. In some cases, though, teachers have a pretty legitimate pedagogical reason for hiding that, that they don’t want students to know how to do the task. And I found this to be the case, particularly in fields where creativity is really important: performing arts, studio arts, even engineering or some STEM courses, where teachers really want students to cast about for a while and kind of use their imagination and see if they can come up with something unique, if not into the discipline, at least unique for the student to try to figure out some new process. And there’s value in that. When teachers want to do that, we did have some pushback from teachers in our original TILT research studies, where they said, “What happens if we don’t want to tell students how to do the work, like part of the task is for them to figure out how to do the work?” So in that case, we asked for those teachers to just say something like, “Part of the purpose of this assignment, in addition to the skills and the knowledge we’ve talked about, part of the purpose is for you to struggle and feel confused, while you invent your own approach to the question.” And we think this is what helps to preserve the student’s sense of confidence and their sense of belonging. Because instead of having that moment of panic of “Oh, no, I don’t actually know how to do this, I don’t even know where to start, I don’t know where the resources are, I don’t know what I’m doing. Maybe I shouldn’t be doing this, maybe this isn’t for me, maybe I shouldn’t be in this major, or in this course.” Instead of going to blaming it on themself or to questioning whether they’re up to the task at all, students can say instead, “I am totally lost right now. And that is exactly where I’m supposed to be. I know I’m on track. I’m doing great. This is the confusion part that comes before the clarity. And I know that because we talked about that and the professor said, this is part of what we expect to happen. This is intentional, this confusion, you’re supposed to feel lost right now.” So that’s sort of what we can say about the task. And the benefit of students knowing upfront what the task is, or knowing how they plan to approach the assignment or the project, the benefit there is that students get to spend 100% of the time they’ve allocated to work on this project, doing their best quality work, and they don’t lose time trying different approaches to see if this or that is going to work or looking for resources that aren’t what the teacher intended for them to be using. Instead of losing time, on the “how,” students get to spend their time so that what teachers receive then is most of the time what we’re looking for, which is “What is the student’s highest capacity right now?” Let’s see an example of the best work that the student can do right now, so we know where they’re at and we can bring them further so that they can advance in their learning. But if we accidentally end up in a situation where a teacher didn’t intend for the students to be confused, they expected the students to take a particular approach that they may have even mentioned at some point in class. So that’s why they think the students know that that’s the expected approach. I don’t want to say the correct approach but at least what they expect students to do. So if we think that students know how to do what we expect them to do, and the students don’t know what we’re expecting them to do, then there’s this chunk of lost time, where what we’re measuring then in the end is what happens after the students spend a chunk of their time lost trying to figure out how to approach the work, and then whatever time is left after that doing their best quality work in the amount of limited time that’s left. So part of the “task” piece of the framework is about what do we want to measure? Right? Do we want to be assessing the best quality work that students can do? Or do we want to be assessing what happens when you give a really varied, diverse group of students a particular assignment to do and you don’t give them 100% clarity about how to do it, and then kind of what you’re measuring is which students have, through no fault of their own, not encountered that information in their lives before coming to this course. And then you also get to identify who are the students that maybe because they had some other kinds of privileges that not all the students had, who are the students that can figure it out faster, because they come equipped with those privileges. So you can begin to see that this is an equity issue. So if talking about the purpose of the assignment kind of speaks to the student’s motivation, and to the value that they will gain from doing the work, and maybe to their ability to assess if they’re getting that value while they’re doing the work, the task speaks to even more of an equity situation where we’re trying to get all of the students to the same starting line of understanding of how to do it, and of having all the resources they would need to do the work to complete the work. And we want to make sure that students are all at that same starting line before they start the assignment. So that’s kind of the equity piece of this. And then finally, the third part of the framework is about criteria. We want students to be able to understand while they’re doing the work, how well are they doing. We want them to be able to make corrections, if they end up with a finished version that doesn’t look like what successful work would look like in this kind of a scenario. But if the students have never seen what successful work looks like, and they probably haven’t, because why would you assign them to do something that they’ve already seen many examples of; they wouldn’t be learning anything new. So kind of by definition, students aren’t going to know what successful work looks like when it meets this or that criterion in the discipline. So what we encourage teachers and students to do there when they’re considering the criteria is to offer students more than just a checklist or a rubric, because the words on a rubric or checklist might mean something different to the student who hasn’t done this kind of work than they do to the teacher who’s really immersed in this kind of work. An example I sometimes offer is, let’s say, I asked students to write up an analysis of a 15th century wooden painted sculpture of the Madonna and child from when I was teaching Italian Renaissance art history courses. In an art history course, the word analyze, like the tasks, the actions that you take when you are analyzing something, that’s a very, very different activity than analyze in the context of an economics course, or in the context of a chemistry course. But if the student hasn’t done this kind of analysis before, you can’t know for sure that they know what you’re asking them to do. So we kind of have to talk that through and students are going to need to see some examples of real world work in the discipline so that they can, with you, in a class meeting, talk about how do we evaluate analysis in this example from the real world, or in that example from the real world. And you won’t find any one example that matches every criterion of the assignment you’re asking students to do, usually, so you need several examples. The benefit of several examples is also that you can begin to talk about the relative success with which different examples are meeting a particular criterion as well. So once we’re in a conversation with students, and we hear back from them, that they’re telling us, what we had hoped they would understand about the skills they’ll practice and the knowledge they’ll gain, that purpose, about how they’re going to approach the work, the task, and about how they’ll know that they’re doing good quality work, the criteria, once we hear students telling us that, that’s the moment that things have become transparent. It is that activity of communication, that conversation with students about purposes, tasks, and criteria, that’s where the transparency comes from. And when we are done with that conversation, we know that students are at the same starting line of readiness In terms of their understanding of what they’re going to do, and also, in terms of their confidence that everyone has the resources that they need, in order to complete that work

Rebecca: What faculty believe is important for students to learn doesn’t always align with the goals of students. Can you talk a little bit about some strategies for bringing these into better alignment?

Mary-Ann: Sure, I think that this kind of speaks to the purpose part of the transparency framework. And often teachers are expecting students to learn something that is very valuable, we wouldn’t spend our time teaching things that don’t have a lot of benefit for students or that they would only use today and it wouldn’t be useful to them later in life. We like to teach things that have value. And so, when we are communicating with students about that value, we’re talking about the skills that students will be practicing. They won’t perfect them on this assignment, but they will begin to strengthen a particular kind of skill set. And they will gain some sort of disciplinary knowledge that can be useful to them later. And we know that sometimes disciplinary knowledge changes over the years as people discover new things and publish new things in any field. Sometimes that knowledge changes. But having some knowledge now does give you important value if you’re going to continue in that discipline or if you want to understand basic principles of a discipline that you might find useful elsewhere. So if students and teachers have a transparent conversation or communication, it could be a written communication, it could be something that they record and put on a website, it could be an asynchronous kind of conversation in an online course. But whatever form that communication takes, I think students and teachers when they’re on the same page about what the knowledge is, what the skills are, that are the focus of this assignment, students will feel more motivated to do the work, because they’ll see that it has benefit for them. And it doesn’t feel like a rote exercise, or just churning out another problem set or another art history analysis paper. There’s some value here that the students know upfront what that value is. And when the teachers hear the students reflecting back to them in this communication, that this is the value that they will be gaining, then we know that students have a kind of motivation to benefit from this assignment.

John:One other issue is that students have come up with some way of learning while they’ve been in elementary and secondary school. But those methods that they picked up are not generally the ones that are most effective. How can we encourage students to adopt learning strategies that they may be resistant to because for example, students, when there have been surveys of what types of learning strategies they found most productive, students often say they prefer to be lectured at, because they learn more from the professor that way. And also, many students don’t like active learning strategies. While they learn more, they don’t perceive it that way. Partly because of those desirable difficulties you referred to before, that when they’re struggling with something, it’s a little bit less pleasant than sitting there nodding and smiling and having everything seem to make sense. How can we encourage students to accept those desirable difficulties associated with learning so that they can learn more effectively,

Mary-Ann: I want to say that this is something that the TILT framework can definitely help us with. And this is not an uncommon phenomenon at all, I even find in my TILT workshops that I do with instructors, that instructors don’t love collaborative learning either. And in fact, many of these TILT workshops that I do will begin with some kind of a research review about “How do we know TILT works? What are the studies and what do they tell us and show us the data?” So we get off on this kind of role, where we’re almost in a traditional lecture format, where like someone’s delivering some information, and people are listening, and then they have questions about it. Or maybe they have challenges to say, “Wait, this doesn’t make sense, let’s talk about this.” And then I kind of switch the method that we’re using. And I’ll ask people to break off into small groups and begin to analyze a particular assignment and talk about where do they see the purposes, the tasks, and the criteria? Before I do that, I acknowledge the fact that we are shifting gears, and that we were doing fine with this sort of Q&A format. You know, look at the research and then think about it and talk about it. Ask questions. Why would I switch that up now? Like we were on a roll, we were doing great. Everybody was sort of on board. Why would I change that now? And so I use the TILT framework to talk about why we’re shifting gears now. What is my purpose in having you use this different method? So if it’s a peer learning method, as it is in the workshops, or as it might be with students in a class, we want to tell students: “Why are we now manipulating your learning experiences this way? Why would I do that to you when I know that sometimes students resist this, when I know that it can be uncomfortable, because I don’t personally always like to do it when I’m in a learning experience?” So if we can tell students, here’s why this is going to benefit you, because you don’t just hear it, but you have to struggle to apply it, you have to fit it not to the situation that I was talking about, where it all sort of makes sense when it rolls over you and you’re hearing it. But you now have to take the principle of what we were talking about, and apply it to this new unfamiliar scenario. And the benefit of that is that you will discover you will hit a barrier at some point in that process, where you will discover the exact piece of information that’s missing for you. You will discover exactly where you hit a barrier to your understanding. And you will have an opportunity right now, right here with me, the teacher in this class, to address that confusing point. And the benefit of doing that now, as opposed to later when you’re doing a graded assignment, is pretty obvious, you get the benefit of having the difficult learning experience in a safe environment that doesn’t lose you any sort of points on your grade. It doesn’t have any negative impact on you the way that it might if you waited until the end of the term to do some massive project and you hadn’t really done a lot of the homework or done a lot of the practices and so you didn’t really know what you didn’t understand until it was kind of too late to do anything about. So I think in short, what I’m trying to say is when we’re asking students to do something uncomfortable, that has a really solid pedagogical reason, that has evidence behind that, it is an evidence-based practice, we want students to know that upfront, because that then will increase their motivation to do it, because they see how they’re going to benefit if they do this thing.

Rebecca: One of the things that students often struggle with is when they start new courses with new faculty, and new ways of doing things and determining what the instructor will expect out of them and out of that learning experience. Can you talk a little bit about how the TILT framework could allow students to shift their focus to learning if it was adopted in the design of the course rather than just an individual single assignments?

Mary-Ann: Yes. And in fact, this is a way that lots of faculty are using the TILT framework, is to think about how do I TILT not just a single assignment, but a whole course. So usually, when people are introduced to the TILT framework, the original ask for all our research studies is would you please do this two times in an academic term, just twice? Because we wanted to see how little change could you make and have a beneficial impact on students’ learning, because small change is much more likely to happen than massive change. But once you’ve made that small change as an instructor, and you see that when you do this with two assignments, there’s some real benefit for students. And on the TILThighered.com website, there are publications by faculty who talk about not just how the quality of students’ work increases, but how the teachers experience in grading, or in responding to students, or in how many students will ask for an extension at the last minute, like these difficulties that teachers often face are diminished, while the benefits for students and the quality of students work increases. So once you begin to see this in the small scale of assignments, teachers then, maybe in the subsequent term, will think about what else could I TILT? Could I TILT in-class activities? Could I TILT a unit of this course? Could I TILT the whole course? And then the effects or the applications can grow. So we can apply this to a single assignment, we could TILT a whole course, we could TILT a curriculum in a department, we could TILT a program, we could TILT an institution’s learning outcomes and thread them through not just all the courses, but through all the co-curriculars too so that students might discover in their work-study job that they’re practicing one of the critical thinking outcomes, that’s a goal for the whole university that connects with what they were doing in their accounting class. And then we can even think about this in terms of a national framework of learning outcomes as well. So there are many scales at which you can apply that to a framework. And one of the things that I’m really enjoying about doing TILT full time, is that I can work with groups of schools, groups of institutions, so not just the Washington State group that I mentioned to you, but several weeks ago I was in the state of Kentucky where working with teams of teachers from institutions across the state, for the whole state system, to think about aspects of how do you map out a path for students to succeed in fulfilling their curriculum? And then how do you pursue that path? How do you complete that path? And in that case, we were using the TILT framework as a strategic planning framework to think about once we know what the plan is, like, once we’ve mapped out our plan for how students can effectively complete their degrees, how do we then communicate the value of that degree, not just to the students who are doing the degree, not just to the students’ families who may be contributing to the costs of doing that degree, not just the costs of the student’s tuition, but the cost of the student not being an earner in that family. And we want to communicate this to all the stakeholders, so the students, their parents, faculty, and staff at the institution, to state legislators who may be voting on packages of funding to higher education in their state, to individual grantors who might be funding particular scholarships. And we want to be able to communicate the value of this degree to every stakeholder in a state system that way. And the TILT framework is very helpful for thinking across multiple audiences, because that’s a pretty difficult task to communicate clearly to all of those different kinds of audiences. But it’s pretty essential for the success of higher education in this country. And so we spent a couple of days using the TILT framework as a strategic planning framework to think about how do you communicate the value of a degree? There are lots of ways that you can apply the TILT framework. Another example is I was working with a school in Texas over the summer, and they were TILTing their entire college success course. Many institutions have that kind of course in the first year, and some of them had TILTed individual assignments. And they decided they wanted to put the team of all the teachers together, and then subdivide that so that a smaller team of teachers was working on each week of the course. And then all the assignments and the lectures or discussions that would go into that week. And then we use the TILT framework as a larger framework to connect that whole course. So that from week to week, the purposes, tasks, and criteria were pretty clear. And students understood the path for all of their learning across that course.

John:Have you tried taking on the Florida Legislature? [LAUGHTER]

Mary-Ann: I have not.

John:That’s a real challenge, I suspect.

Mary-Ann: Yeah, I have worked with schools in Wisconsin. Last week, I was working with a school in Tennessee, right after a couple of their legislators were expelled temporarily. This kind of a framework, I think, can be effective in a lot of different higher education systems and contexts. That’s one of the beauties of it. Because this is something that teachers can do, starting right now, to complement any kind of larger, institutionally driven or federally funded program that might focus on student success. A lot of the time, those programs don’t necessarily feel like they’re directly connected to what faculty members are doing in the day to day in their classes. But using this TILT framework is something that you can do that will advance students’ success that will then make you feel more like you’re connected to these larger ongoing efforts that might be focusing on something that you don’t do directly, like targeted scholarship funding, for example. But that’s part of the beauty of the TILT framework is that it can work in many, many different contexts, and across different scale sizes of projects, as well.

John:And it works nicely for faculty because you end up getting work of the quality and the type that you expect, rather than getting student work that you find disappointing. And similarly, students end up doing work that they’re much more happy with, because they were not guessing at what the instructors want. So it just seems really, really logical. But it’s not always so widely practiced. Your efforts are really helpful for all of this.

Mary-Ann: I think one of the reasons why people might be hesitant to use the TILT framework, you don’t necessarily want to try doing something different that could suck up time that could take time away from delivering important content in the course, and what teachers have discovered and written about and published in the National Teaching and Learning Forum and other places you can see on the TILThighered.com website, what teachers have discovered is that if you take some class time to talk about the purposes, tasks, and criteria for a project before students do it, by the time that practice is completed, everyone has saved time; that time gets recouped, and students have learned a larger quantity of what we had hoped they we’d learn because when we deliver content in a course, we don’t know that students are absorbing it the way that we’d hoped or that they could apply it the way that we’d hoped. So I think by the end of the course, if you’ve used the TILT framework a couple of times, you’re in a situation where you’ve worked in a way that is more time efficient, somewhat, and you arrive at a place that, as you say, is more satisfying for students and teachers, because more of the time has been spent with the students doing the highest quality work possible.

Rebecca: I think one of the things that can be challenging for faculty initially is that if you’ve never communicated in this way, it’s hard to do it the first time, because anything you do the first time is difficult. But once you have a little practice doing it, it’s easy to adopt and expand across a course or across a set of courses.

Mary-Ann: That’s so true. And I think that the way that we’ve structured the TILT framework, it looks so simple, it’s a three-part framework. Applying it then gets you into some complexities that are important to clarify. I think you’re absolutely right, the first time we try anything that’s unfamiliar, just like for students, it’s more difficult. And then we kind of get the hang of it. And then it comes much smoother, and much easier. The TILT framework for starters, is pretty simple. It’s got three parts, right? And I think you could probably share a link to the one-page version of the framework that we give to students, that sort of spells out the framework: purpose, tasks, criteria, the knowledge and the skills. And then at the bottom, there are some of the evidence behind why we know this works and some footnotes, so that students can see on one page, this is a real thing. It works, it helps you. It is, in some cases, equitable, and it is probably worth giving it a try. And if you can see all that on one page as a student, then you might be more willing, especially in a context where a teacher is describing to you why this will be good for you, why this is a benefit for all of us. And then for teachers who have not encountered the TILT framework, when students can bring in this one pager that has some studies listed at the bottom and footnotes, they can see that when the student is asking me, why should I bother? This is actually a legitimate question. This is not a troublemaker student, this is a student who actually knows that they will benefit from knowing a little bit more in advance about this assignment that they’re planning to do. So we try to make it as easy as possible to implement. And then we also try to say only a little bit of this will make a statistically significant difference for students’ learning, so that you only have to try it a couple of times in a whole term. And you’ll probably see the kind of differences that we saw in terms of increases to students’ confidence and their sense of belonging, and their metacognitive awareness of the skills that they were practicing and developing. So if you’re doing anything new or different for the first time, yes, there’s some difficulty to that, but this one is a very, very desirable difficulty. [LAUGHTER]

John:We’ll share a link to that one-sheet document as well as to your website in general. And you do have a lot of research cited on your website. And there’s also some ongoing projects. Could you talk a little bit about those?

Mary-Ann: Yes, we are sharing all the resources that we possibly can on the TILT higher ed website, because we want for everyone to have access to this. Some of the places that benefit most are places that might have the least amount of money that is allocated for faculty development or educational development. So we want to make sure that this is accessible to anyone who would want to try it. And then the studies that we’ve done in the past, there are a few studies that have indicated to us a number of the benefits of TILT. One of the first studies we did was the national study we ran with the Association of American Colleges and Universities. It was funded by TG Philanthropy and my colleagues working on that project were Tia Brown McNair and Ashley Finley. And what we did there was we worked with a group of seven minority-serving institutions from across the country that represented every possible type of minority-serving institution, as well as a range of educational contexts like urban and rural, two-year, four-year, research university, really small in scale, large, residential and non residential campuses because we wanted for teachers to look at our results and see, “Oh, well, this worked for those faculty at that institution, and there are students like my students in that mix, so maybe this would work for my students. And in that study, we started with 35 professors at seven schools and we surveyed about 1200 students and we saw that, for the students who received the more transparent instruction, their competence and their sense of belonging and their metacognitive awareness of the skills that they were developing, those increased, those were higher for the students who got more transparent instruction than for those who got less transparent instruction. And then we also saw in that study some differences that showed us that while all the students were benefiting to a statistically significant level, underserved students were benefitting slightly more. So first-generation students in their family to attend college and ethnically underrepresented students and low-income students have slightly larger benefits than the benefits for the whole group. And then in our second study, we focused on how long does this effect last. So we worked with a group of University of Nevada – Las Vegas students. At the time we were working with that study, University of Nevada – Las Vegas had the most diverse undergraduate student population in the nation, according to US News and World Report. And we know from other studies, like Walton and Cohen’s, 2011, Science Magazine article, for example, we know that when students’ confidence increases, when their sense of belonging increases, they tend to persist longer in a course. So in courses that have higher levels of confidence and belonging, fewer of the students would drop the course, for example, more of them more likely to complete the course. And we wanted to see how long does that last. Is it just that course? And some studies indicate that this could last for a year. And what we did was we kept looking at the retention rates of these students to see how many of them were still registered a semester later, a year later, two years later. And we saw that by the time students were in their third year of university as undergraduates, those students who had received transparent instruction in one of their large gateway intro courses in their first year, those students were a little bit more likely to be still registered as students in their third year. And we’re now tracking that out to six-year graduation rates. So we saw that not only does transparency have a beneficial effect, it’s statistically significant, but that effect lasts for a good long time. And then in the state of Washington, we’re now writing up that study I mentioned with the Community and Technical College System. And I think that TILT is particularly helpful in that environment, because the population of community colleges and technical colleges is a little bit more diverse. And we have more students who belong to that underserved category of students, first-generation, low income, ethnically underrepresented. And what we’re finding from that study is we’re understanding a little more about how does transparency work, and I want to thank all of the researchers who are contributing to all of these studies too, because I’m not an educational statistician, so Daniel Richard, and Carolyn Weisz and Kathryn Oleson are contributing to this study and doing a lot of the analysis, along with help from some graduate students who have been working on this project over the years. What they’re discovering is that transparent instruction has a direct impact on students’ awareness of the skills that they’re learning, and it has a direct impact, similarly beneficial, on students’ sense of belonging. And then separately, sense of belonging has a direct impact on students’ metacognitive awareness and skills that they’re developing. So TILT has this direct effect. And then there’s this other effect between belonging and skill development as well. So we’re finding out more about precisely how TILT works for the benefit of students in these studies. And I think in terms of next studies, I want to be asking questions that really matter to populations of faculty and students around the country. So we open up the TILT research team to anybody who’s curious about this, and a number of faculty have asked about, can we say something more about how this works in an online setting, in an online synchronous setting in an online asynchronous setting, and we’ve got a few publications up on the website about that, but others are looking at that a bit more. And then we have another person who’s looking into just the impact on low-income students to see if we can find out more there about the details of how this works. And I’m really curious to see if we can work with large state systems, what can we find about the most time efficient, most beneficial ways to apply transparency and learning and teaching in community college settings. And I’ve also noticed that as I begin to do more work internationally, because I now have more flexible time to be able to do that, the colleges of applied sciences, like in the European Union, for example, they have a kind of three-year degree that is similarly focused on students’ learning something from their degree like they do here in a community or technical college that will lead them on a path into sustainable long-term employment and a career. So I think that this is going to be a really beneficial place to focus TILT efforts and to do some more research about how can we long term have an impact on not just students’ education, but how that is a pathway into a career. And I’m hopeful that we can find out more about that, like the longer long-term effect of TILT. But I’m also really open to inviting anyone who wants to do more research with the mountains of data that we’re sitting on, to discover something that is of interest to them about how students are learning, and how we can help students succeed more.

Rebecca: I really love all the resources and examples and research materials, worksheets, that are on the website. They’re really handy for folks who are starting out. We always wrap up by asking: “What’s next?”

Mary-Ann: What’s next for me, and then what might be next for teachers and students too. So we’ve talked a lot in detail about how TILT works, and how we know it works, and what more we want to discover about how it works. But I want people to remember that this is really a small effort, it’s a very easy lift that has a really large benefit from the size of that lift. And so I would really encourage teachers and students, if they’re going to do anything at all, even if they have no time to adjust any assignment prompts or to adjust anything about the way that they’re teaching or learning in a classroom. If you use any one single thing, I would say use that framework that we built for the students that has the footnotes at the bottom, and it’s called the “unwritten rules” and that framework, and I think you could probably provide a link to it, that’s what I would hope people would do next, just take that framework with you to anywhere that you’re communicating with your students. And the students will tell you how to make the work more transparent for them. Ask students what they see as the purpose, the task, and the criteria. And you’ll discover very quickly, very efficiently, how you can make that work more transparent so that all students are starting to do the work with the same understanding about what’s expected and with the same set of resources that they need in order to do it. So that’s what I hope is next for teachers and students.

Rebecca: And I hear all the faculty cheering about efficiency, and quick. [LAUGHTER]

Mary-Ann: That’s good. Yeah. So that would be the most time efficient thing to do, I think is to have students teach us more about how to be more transparent. And then in terms of researchers, I’m hoping that researchers will think about what can we learn more about? Can we learn more about what motivates students? Or what forms students’ sense of belonging? Is there anything in our survey data that would shed light on any kind of work you’re doing around that? Is there anything in our survey data that would shed light on more of the research on neuroscience and how that’s impacting learning? Or is there anything in the research that we have in our survey data that might help clarify what would be most beneficial for the very most at-risk students? So if we look at federal government statistics, National Center for Education Statistics about retention rates and graduation rates of different populations of students? Can we double down and look at those students with the very lowest graduation rates? And can we find something about TILT that would be the most beneficial for that population of students? To me, that’s a really important and interesting question. And then I really do want to be finding more locations where TILT could be useful, small scale for teachers and students, large scale for state systems or national systems to be thinking about how to apply this all for the good of students success, and for the satisfaction and time efficiency for teachers work as well.

John:If you’re finding these results of long-term persistent effects from just a single intro course, imagine what would happen if all intro courses use the TILT approach. I imagine the effect would be magnified if it was adopted at a broader level and it is being adopted at many institutions at a broader level.

Mary-Ann: I absolutely agree with you that applying TILT across the largest introductory gateway required courses at any institution would be probably the most efficient way to improve retention and graduation rates. Because if you go for the largest group of students as they enter, and you reduce the number of those students who might be thinking or doubting or wondering if they should continue, and if you increase the number of students who feel confident, who are aware of the value of what they’re learning, in terms of skills and knowledge, and if you increase the number of students who persist from the first year on, then that’s where you’re going to have the best success in increasing retention and graduation rates. I agree with you. I think that’s a really strategically wise place to invest TILT effort.

Rebecca: Well thank you so much. We’re looking forward to sharing this with our listeners.

Mary-Ann: And thank you so much for the opportunity to talk with you this afternoon, I really appreciate it

John:Thank you for all the work you’re doing.

[MUSIC]

John:If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

286. Grading for Growth

Traditional grading systems provide incentives for students to focus on maximizing their grades, rather than their learning. In this episode, David Clark and Robert Talbert join us to discuss alternative grading systems that encourage students to recognize that learning from mistakes is a normal part of the learning process.

Robert is a Professor of Mathematics at Grand Valley State University and the author of Flipped Learning: A Guide for Higher Education Faculty. David is an Associate Professor of Mathematics, also at Grand Valley State University. Robert and David are co-authors of Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, which will be published this summer by Stylus Publishing.

Show Notes

  • Talbert, R. (2017). Flipped learning: A guide for higher education faculty. Stylus Publishing, LLC.
  • Talbert, Robert and David Clark (2023, forthcoming). Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, Stylus Publishing.
  • Nilson, L. B. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Grading for Growth blog
  • Robert Talbert’s other blog

Transcript

John: Traditional grading systems provide incentives for students to focus on maximizing their grades, rather than their learning. In this episode, we discuss alternative grading systems that encourage students to recognize that learning from mistakes is a normal part of the learning process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Robert Talbert and David Clark. Robert is a Professor of Mathematics at Grand Valley State University and the author of Flipped Learning: A Guide for Higher Education Faculty. David is an Associate Professor of Mathematics, also at Grand Valley State University. Robert and David are co-authors of Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, which will be published later this year by Stylus Publishing. Welcome, Robert. And David.

David: Good to be here.

Robert: Thanks for having us.

John: Today’s teas are:… are either of you drinking tea?

David: I am.

Robert: I am drinking tea. I’m more of a coffee guy normally, but I figured for the occasion I would bust the tea out.

John: What type of tea are you drinking?

David: I have, because its afternoon, a really lovely almost white jasmine right here.

Rebecca: Nice.

David: …really delicate and anybody who cares about tea is going to be horrified that I sweetened it with honey,

Rebecca: But it is in a really nice polka dotted mug. It is a lovely mug. Thanks.

Robert: And I have cheap stuff from the grocery store, because that’s my brand. This is a Bengal Spice by Celestial Seasoning, and my wife and I are addicted to this tea. We drink probably four or five cups a day of it, though to be honest when it’s cold out, but it usually is [LAUGHTER] here in Michigan.

Rebecca: And I have Jasmine dragon pearls today.

David: Oh, nice choice. Both Jasmine.

John: And I have spring cherry green tea. It’s a very cold wintry day here as we approach spring in upstate New York. We’re recording this a bit before it’ll be released, so by the time you hear this, we should be having the beginning of spring both here and Michigan, I’m hoping.

David: We hope, yeah..

Rebecca: Your tea choice is definitely a big wish for spring. [LAUGHTER] So we’ve invited you here today to discuss Grading for Growth. Can you talk a little bit about how this book came to be?

David: I don’t remember, Robert, if we were in person, or on a Zoom call or something, but I remember you saying at some point, I got a phrase for you: academic book about alternative grading. And I said, “You know, I need a project for my sabbatical.” And then I think it just happened from there [LAUGHTER], and so…

Robert: Yeah, we were in person, David. David and I, we’re not on the same hallway, we’re kind of on opposite arms of a T-shaped hallway. And so we run into each other, sometimes literally, at the intersection of these two hallways, and that happened one day. It was pre-pandemic, I guess. And it was just like, “Hey, Dave, I have this idea, Somebody should write a book about alternative grading. Because David and I have been using various forms of alternative grading in our classes for a while. And he was like, “Yeah,” and that was it. [LAUGHTER] But I think we were also tapping into some stuff that we have been hearing and around our Math Department and elsewhere, our colleagues outside of Grand Valley that started growing even three years ago, just a growing interest or dissatisfaction with the way grading is working. And we’ve been trying some stuff and thought, don’t you know, the best way to make change is to just get your ideas out there. And so this seems like the right place, right time for us.

John: So what types of alternative grading systems do you discuss in this book?

David: A pretty wide variety. And actually, something I’ll say, I think, both Robert and I tend to try to avoid labels too much for these, like there’s useful names to describe different approaches. But we do things like standards-based grading, specifications grading, ungrading, standards-based assessment. standards-based testing, but we look at a really wide variety of alternative grading approaches. And we’ve actually tried to come up with a framework that sort of describes what their common elements are, so that we don’t have to worry as much about names as what the useful features of them are. We found that people can really get locked into an idea of what, for example, upgrading means, and it’s not always a super useful thing to argue about with them.

Robert: Yeah, I would echo that too. Being mathematicians, Dave and I are both really into abstraction and so we look at these specific things that we see, but we’re more interested in the big, general overarching unification principles like what are all these models that are all good, and all applicable, in different places, to different levels of success, and what do they all have in common? And we do discuss ungrading, we do discuss specifications grading, standards-based grading, and a whole lot of approaches that are kind of in the in-between interstitial space, mostly through other people’s stories. I think the heart of this book and David’s real amazing contribution, what you really spend your sabbatical doing, Dave, was interviewing dozens and dozens and dozens of actual real life frontline professors, nobody is in one camp exclusively, everybody’s using some kind of combination, some kind of mix of all these different ideas. And so I think, to me, what the real contribution that our book makes is showing how different things can look, you can start from these basic building blocks, but real people with all kinds of different classes and life situations and professional situations are making this work by listening to their students and adapting appropriately.

David: I can’t emphasize enough how much variety there is in the people we’ve interviewed and the disciplines that they’re in, the types of classes they’re teaching, like it was an amazing thing to talk with all these people and see, okay, they’re able to use different types of alternative grading but absolutely across the board: in labs, in huge classes and tiny classes, and upper-level classes, and absolute freshman-level intro classes. And so it’s been just fantastic to hear how everyone’s doing this and try to put it together into something that could work for anyone, that everyone will find something useful.

Robert: Yeah, no two of them are alike either. And what’s even more amazing to me is that we had to cut a lot of those case studies. And we almost have enough material for a second book just for the case studies. We’re going to keep it on the blog, I think, unless Stylus wants us to do it. But, I mean, there was a lot out there that we don’t talk about in the book. So there’s more where that stuff comes from.

Rebecca: You mentioned that these alternative methods have some common themes or common threads. And maybe it would be helpful to talk about those common threads in relationship to the problems with traditional grading that lead people to these other alternative methods. Can you talk a little bit about that?

Robert: Well, I think one of the places that you begin to see some of the shortcomings of traditional grading is just looking at the history of where traditional grading came from. And it’s sort of a weird and checkered, and very short, history. Many of your listeners might be surprised to know that the current system of points based A-B-C-D-E-F, 4.0 GPA type of grading system is only about 100 years old. It’s a relatively recent innovation in higher education. Higher education, formally, is about 1000 years old. 1088 was the first university and the first readily identifiable grading system that we know of now is like 1890s. So it showed up very, very late to the party. And immediately it was co-opted by Industrial Revolution era approaches to teaching and learning which treated students more or less as subjects. The word “grading” really comes from agriculture, when it comes to like grading Grade A beef and grading grain and flour and that kind of thing. And pretty soon it became entrenched. And we have a situation now where when you look at where grading has led us, it’s highly questionable whether grades really do what we want them to do. And it seems almost certain that they do some things that we don’t want them to do. They lead to issues with ranking and sorting students, pitting students against each other. It’s not clear to me that the statistical validity of points-based grading is even well established. I mean, we put points on things, but they’re not really numerical data in any sense. Computing an average of points across a system of exams does not necessarily tell you how much a student knows. And that, to me, in my view, is the fundamental issue that I have with traditional grading, why I moved away from it. I just didn’t feel like the data was telling me anything.

David: I’d like to jump in on that one, in particular. A thought experiment that I think is really helpful for anybody to do is, let’s say you give a big exam, and a student takes it and they get 60%. So, most systems, that’s basically failing. What does that 60% tell you? What did they know? What don’t they know? Why did they do that poorly? Did they actually do poorly? Do they know 60% of the things that you’ve covered on that exam really well, and the other 40% not at all? Do they have a mediocre level of understanding of everything? Maybe they understand absolutely everything great, and they had a terrible day, and they had to get the kids to Grandma’s house or their work had to keep them late. Maybe they’re sick. All those things are wrapped up together, in that 60% and you just don’t know what it means. And then to add to that, if that student learns, and they really show great effort, and they can tell you later on, “Hey, I’ve got all this stuff, and let me show you how, maybe on like a cumulative final, that 60% is still averaged in and it’s going to permanently weigh them down and their final grade won’t represent what they actually came to know.

Robert: Right. And so grading really cuts against the very process that humans engage in when they learn anything. When I’m learning how to play a song on my bass guitar, and I play it once and I do terribly at it, if I eventually learn how to play it, people should not be looking back at the first time I tried it, [LAUGHTER] they should be looking at the last time I tried or the maybe the best time I tried it. That’s not how recording artists record. It’s not how athletes are ranked, and so forth. And it’s all because of this sort of Industrial Revolution era routes where we have everything measured and sorted out and put together and it inhibits growth. It sort of poisons our relationship with students, it leads to all these extrinsic goals that students now have. Rather than focusing on learning and growth, students get the idea that it’s wrong to fail at things the first time, whereas it’s actually normal to fail at things the first time and then grow from it. And I guess that’s where the name of our book came from, I forget where that phrase “Grading for Growth” popped up. It was way before we were asked for a title for our book. But it’s like that’s what we really want. We want to have a system, even if we must call it grading, we want it to be a system evaluating student work that focuses on and encourages growth as you are learning because that’s one of the great things about being human is that we do grow and we do learn from our mistakes. So where is it in our assessment systems? That’s kind of the fundamental question we’re asking.

David: So it might be helpful if we talk a little bit about this four pillars framework that we have in terms of where we go with that. We kind of spent a while trashing traditional grading here, and I could do that all day. So what did you do instead? What are the things you want to do instead of these things you don’t? And so from my point of view, the most important thing I want to know out of a grade is what does the student actually know? What do they learn? And so, as I said, we have this four pillars model, we talked about four pillars that any good rating system should have and one of those is really a clearly defined description of what it is you’re assessing. So we call it clearly defined standards. I don’t want to catch on to the name standards too much there, but a description of what it is you’re assessing and what it is that matters in that assessment. So how do you know that the student has achieved it? And what is it that the student has achieved? So in that way, that takes care of this issue of what does that 60% mean? 60% of what? And if you’re grading based on a specific thing that you’re testing a student about, then you can say, okay, maybe they’re achieved that or they have not achieved that specific goal. And then we try to incorporate in our other pillars, this idea of feedback loops. So humans work on feedback, they work on trying things multiple times, they work on trying and failing, and having to come back again, and that it should be the ultimate level of what they know, that really matters. So focusing on feedback, rather than focusing on numerical rates. So focusing heavily on feedback, on what’s happened, and how that relates to the standards or the specifications, or whatever the description is that they were trying to reach, and then making that feedback actionable so the student has a chance to actually act on that, and either through a new attempt show that they’ve got that idea, or through revising previous work, show that they come to understand it in a way that counts fully, so that they’re not penalized for needing multiple attempts, just like Robert was talking about with his bass, we’re not going to look at well, the first time you tried, it didn’t work out, too bad. We want to know, ultimately, where were they, so we shouldn’t penalize multiple attempts at understanding something. And finally, it feels like a technical thing, but it’s actually a really big move, moving away from points or percentages and instead, if you’re gonna put a grade on work at all, to make that grade something that’s sort of a descriptor of the feedback, something that basically says, you’ve met this standard or you’ve met these requirements or you haven’t and gives the students an idea of where to go next: you need to revise this, you should try this again, you’ve made it, something like that.

John: I know you’ve experimented with a variety of alternative grading systems. Could you talk a little bit about how you view some alternative grading systems in your own classes?

Robert: Sure, I’ll jump in on that. So around 2017, I described this in the book in my origin story where I came from with this. I had just taught a calculus class and I had just a terrible experience with grades. One student in particular, a fantastic student, she was the epitome of what we’re discussing here. She was on a 10 day delay from the rest of the class and after those 10 days were up, she’d be at the top of the class, but the exam is today, and so her exam scores just went down, down, down, down, down. It was just an awful way for a student to experience my subject. And so I started casting about and I read this book that had just come out by Linda Nilson, who is a legend in the area of professional development, and my former boss at Vanderbilt University. And it’s called specifications grading, and I said “What is specifications grading?” So I read this book, and I was converted on the spot. And I’ve been using specifications grading in almost all of my classes, except for one instance of ungrading last year, ever since. And so, I’m in various forms, and always tweaking, always experimenting, and trying to change things up. But, specifications grading is exactly along the lines of these four pillars that David is mentioning. There are clear content standards that we call specifications. The idea is not to give like points, but marks, to use a sort of a non-American term for those, marks that indicate progress, like you’ve met the standard, that’s what you get instead of a 10. Or you’re progressing, that’s what you get instead of a five, and giving lots of helpful feedback and especially letting students retry things without penalizing them for retry. To me, that’s like the thing that drives the loops. That’s been my main sort of area ever since for the last several years,

David: I’ve used a pretty wide variety of different alternatives. It’s come to the point for me where I think about all these different types of alternative grading systems that have names like standards-based grading, or specifications, or ungrading as elements to put in and they may be appropriate in some classes and not in others, depending on a whole bunch of factors. For example, I’ve taught intro level classes where I use a lot of standards-based grading where that means I’ve sort of divided up the topics into fairly fine-grained standards describing what a specific skill is, and what it would look like for a student to achieve it. And then the grade is essentially based on have they shown me that they thoroughly understand that, they can do the thing described in each of these standards and grades are based on meeting or not meeting those standards. And that can look like pretty traditional assessments, quizzes or exams or homework or such things. But instead of getting an overall grade, they get a mark, like Robert was talking about, for each standard. Yup, you’ve achieved this. Nope, you haven’t yet. Maybe they need to do it a few times. But that’s one element. It’s really useful for sort of discrete skills, intro level things. I’ve used specifications in a similar way that Robert is talking about. One of the strengths of specifications is that you sort of look at a student’s work holistically and say, “Have they overall shown me they can put these ideas together, understand the concepts, use all the different things that I think matter within this assignment,” and that works really well on written work… We’re mathematicians where students write proofs, it’s a detailed explanations of why something’s true, or in a project or in a portfolio or something longer than that. It’s really useful, maybe in upper-level classes or in places where I want students to show synthesis. And I’ve gotten pretty deeply into ungrading, or at least as I use the word ungrading, meaning removing grades entirely, even marks, in the way we’ve been talking about as another element that can be useful either in an entire class or just as individual things to do within some assignment. So I’ve covered a lot of this, I continue to push on what’s most appropriate in different classroom situations and different student situations.

John: I’ve tried some of the same things too. But one concern from a lot of faculty who have tried mastery grading systems or mastery quizzing and those types of things, is that it can put a lot more work on the instructor to do the grading on multiple attempts. In Linda Nilsen book, she suggests giving students a certain number of attempts, but limiting those. How do you keep the work manageable so that you’re not spending all of your time grading additional attempts, as students are working towards mastery.

David: Yeah, the thing you mentioned from Linda Nilson’s book, attempts without penalty doesn’t mean attempts without limits, and that is really important. And yeah, limits like that are one way to do it. There can be limits on number of attempts, or on frequency of re-attempting something or revising something once per week, once total, something like that. Those sorts of limits can just be really useful. And I know, I always encourage new instructors or people just starting to use these systems to start out limited, and then add flexibility because that way, they will be able to do it if they need to, and not do it if they don’t want to.

Robert: Yeah, you know, another thing too is, not just in the reassessments, but in your overall design of your course, including the grading system, you got to keep things as simple as humanly possible. This is something David and I just hammer home constantly, like every other paragraph ends with keep it simple, because people who want to mess around with grading systems are people who like tinkering with systems, and I am one of those people, and I know that when you start tinkering, it’s very hard to stop. And you end up with this massive Rube Goldberg like device that is your class and to you, it looks beautiful, but when you face it towards students… I mean, the first time I did specs breeding, I had 68 learning objectives that students had to meet throughout the semester and it was a nightmare. At the first of the semester, I thought, I’m a genius. I mean, look at this beautiful grading system I’ve devised. It’s going to revolutionize everything. And the only thing it did was give me a grading jail for four months. Just keep it as simple as humanly possible, and you’ve got to cut the work off at the source, I think. David and I wrote a book about grading, but we don’t like grading. I mean, nobody really likes the process of it. But it can be made at least a fulfilling undertaking if you are orienting it towards growth. But you still don’t want to be doing it 8-10 hours a day. So keep things as simple as humanly possible and put some limits on that. It’s okay to create a little bit of scarcity when it comes to the reattempt side of things. The feedback needs to be helpful that you give to students, but it doesn’t have to be incredibly lengthy. It might just be a couple of sentences, it might just be: “You did this really well, and this needs work.” …to kind of keep it right to the point, and then that helps the students who are on the other end of this too also experiencing a workflow possibly overload. And I worry more about them than I worry about myself. And so this makes it more likely they’re going to read your feedback if it’s shorter, more concise, and it makes it shorter to give.

David: Yeah, I want to emphasize that it can be a lot of work for students as well. Something that I suggest a lot is, if you want to use a system that emphasizes these pillars, especially reattempts without penalty, you’ve got to then think about those re-attempts as part of the regular workload of your class. Really what it means is you’ve got to think about the process of learning, that whole process, is part of the time that students are going to have to invest into it, and that you need to value that in your timeline for the class. So I like to think of maybe if you expect a student to work, however many hours per week on your class, account for whatever time each assessment takes, add another half for how much they might need to do revising or reassessing or reworking on things. And that can lead to really difficult decisions. You’ve got to cut stuff out if that’s going to happen. I don’t know, basically, anybody who looks at a classroom teaching and says, you know, there’s not enough in here. And so you got to make those difficult decisions, but it is worth it. Because what comes out of it is a student knowing things that are still in there way better than they would have otherwise.

Rebecca: I think one other barrier that faculty sometimes face is learning management systems, and how they guide you towards particular kinds of grading systems and evaluation systems. I wonder if you can share some tips that you have from your own experience of dealing with a learning management system to help you and your process rather than getting in the way of implementing some of these techniques.

Robert: We’re a Blackboard campus, and we’re making a transition right now and I’m honestly not sure how the new system handles those. But in at least the next to most recent version of Blackboard there was a thing called a schema, which is basically a way of going into your gradebook and defining a way to map number inputs to text outputs. And so I can set an assignment up to be graded on zero or one point. And then I can only enter in zero or one. But I could tell Blackboard if I enter in a one put “meets expectations” in the gradebook, and if I’ve put a zero put “does not meet expectations” or a happy face emoji and a frowny face emoji or whatever I feel like doing. And I have zero experience with any other learning management systems over the last 15 years, so I can’t really speak to that. But I know that many memory management systems allow you to customize the way that your numerical inputs present themselves in the students’ gradebook. And if you can find out how to do that, that’s the way to go, as far as I’m concerned.

David: And even if you can’t, the thing Robert mentioned, that essentially everything is a zero or a one, these systems all want to use points. So just refusing to play that game by making everything a zero or one. Even if students are seeing the zero or the one, it’s a lot easier to interpret the zero or one as success or not, than it is if you have points showing up somewhere else, and you’re trying to convince students No, no, no, when I put in a five, that meant something that doesn’t mean five points, that’s not going to fly. So limiting it to just a zero or one, a complete or incomplete, something like that works pretty well. That’s also true for spreadsheets. So if you just want to keep your info in a spreadsheet, they still want to work with numbers for the most part, but you can think of everything as a zero or a one. And then you start to change your mind from averaging or totaling things to counting. So I’m counting how many standards a student has completed, I’m counting how many assignments they’ve met the specifications on. And that’s really the same as saying, how many ones do I see here? One last thing is, everybody seems to have a different LMS. Even those of us who use the same one don’t quite use the same version of it [LAUGHTER]. So the best thing to do is to find someone who knows it really well and talk with them and say, “How do I make this type of thing display?” …and they probably know a way,

Rebecca: I’m gonna have this conversation with John about our LMS.

John: We’re using Brightspace from Desire to Learn, and it does have those capabilities.

Robert: And one thing I would say is that these days, as the ideas about alternative grading get more and more airplay, I think the tech companies are starting to listen to these things. I mean, you’re actually starting to build these things in as a competitive device like “Ah, you should adopt our LMS, because we can let you do ungrading and you don’t have to deal with all the hassle of hacking your own LMS. So I feel like ed tech companies should be paying attention to all this great interest that’s swelling up, it’s like this is a way to earn customers, honestly.

David: Something I’lll say most LMS’s do seem to do pretty well is to allow you to get feedback in a variety of ways. And so if you can de-emphasize that grade portion, just 0-1 or hide it or something, especially like I accept all assignments through Blackboard nowadays, giving feedback’s pretty easy on that. So that, in my mind, is a benefit.

Robert: Yeah, Blackboard does a really well, actually, that if you submit a PDF into Blackboard as an assignment, it’s easy to leave comments on it, and it keeps all the versions of it. So you can go back and easily see the students trajectory from the very first draft that they do all the way to the present day. And I think that’s really powerful, actually.

Rebecca: I’ve used rubrics to set up essentially specifications as well, like it’s met or not met.

David: Absolutely, yeah. If you know what you’re doing setting up rubrics, I don’t, that’s the problem for me. But if you know what you’re doing, that’s fantastic and it can also save you time.

John: Since the pandemic, there’s been an explosion of interest in ungrading. Does this approach work as well in all contexts, and for all students?

Robert: People are starting to use ungrading to mean anything other than traditional grading. And I think that’s leading to a lot of confusion, honestly. I was working with some teachers recently who were saying that, “Oh, I’m using ungrading but don’t have time, I’m being overwhelmed by all these quests for reattempts.” And it turns out, they weren’t using ungrading at all, they were totally grading thing. When we say ungrading, we mean like a specific approach to evaluating student work where nothing gets a mark, literally ungrading. You get a letter grade at the end of the semester, because most places require that, but it’s all based on a term that’s come up recently, that I really prefer, is collaborative grading. You’re working together with the students throughout the semester to talk about the quality of work, but nothing is getting a mark on it. I have a comment for you and you can either choose to iterate on that comment with a feedback loop or leave it alone. At the end of the semester, you’re going to sit down together and collaboratively determine what your course grade is based on the body of work that you’ve accumulated in some sort of portfolio situation with some clear standards for what an A would look like, a B would look like, and so forth. So that’s what we mean by ungrading. The explosion of interest in “ungrading” may or may not be all about ungrading the way we’re referring to it. So I just want to be clear about the terms before we answer this question.

David: And just add even another thing, it’s both that sort of umbrella term and what Robert just said and the general philosophy and a buzzword that people attempt to apply to random things they use. It’s unclear what someone means when they say ungrading.

John: When we’ve been talking about it, we’ve been talking about it exactly as you described as an extreme form of alternative grading where there are no grades other than one that is decided in consultation with the student at the end of the semester. And sometimes at midterm if that’s required in the institution. From what we’ve seen, this is used quite a bit in humanities fields, but it tends not to be used very much in the STEM fields. What are some of the barriers or some of the concerns that might be raised for ungrading in terms of in what contexts it may not work as well.

Robert: I have tried to fully ungrade a course once, and I might use it again. But the issue that I was coming up with with my students, is what I say is that ungradiing works precisely as well as students’ ability to self assess. If you have students who are struggling with the ability to self assess, or they’ve never tried it, or they’re new students, or younger students who maybe are still emerging with that idea… the whole thing about ungraded is that it’s predicated on students looking at their own work and self evaluating. And if that’s an issue, then I’m not totally sure that ungrading, as we’re defining it, best serves that population. I know I struggled with it. And this can be more me than anything else. Maybe I just don’t know how to draw students out and I need to work on that. It’s an area of growth for me, certainly. But if you have a population of students, for whatever reason, whose ability to self regulate or self evaluate is questionable, or kind of low, we’re not going to say bad students, because we’re all growing… it’s the whole point of our book… if you deploy ungrading with that population, you might be sort of targeting the weakest point in your class. And so maybe, in that case, some marks wouldn’t be such a bad thing, like use a specifications grading approach where the marks are just like: you’re progressing or you’ve met the standard. Just some kind of simple mile marker that shows students where they are. When I used this, once, it was an upper-level class, it did have a lot of writing in it, a math class. I would leave comments on their paper and I would just get questions like, so does it meet the standard or not. And I didn’t like to sort of be telling students this, but I felt like this is the way I helped them. I said like, “hasn’t met our standard yet, here is what you need to do.” So I thought, if I’m going to do that, why don’t I just put a mark on it? It seems like I’m just beating around the bush and trying to be cool by not giving marks and it’s like I’m thinking more about myself than I am about the students at that point.

David: I think that a general thing that everybody needs to think about when using any kind of alternative grading is, this is something new for students… almost guaranteed. And like any other new thing, it needs time to learn, you need to encounter it a few ways, you need to try to make sense of and probably fail initially, and then come back and you’ve got to work to understand it. Ungrading is sort of an extreme form of that. There’s more that needs to be understood about how it works, there’s more skills that need to be built. And I think, as instructors, we need to think about that and think about teaching how our evaluation systems are working and how the grading is going to work. And think about what skills we need students to build up in order to successfully engage with those. And ungrading asks a large amount of that. And I say this, as I very much enjoy using ungrading also in some upper-level math classes and I think it’s worked pretty successfully for me. But this sounds funny to say about something called ungrading, it requires scaffolding, it requires helping students build up that ability to self evaluate, advocate for themselves, to be able to understand what it is that matters in that particular system. But what I just said is also equally true for standards-based grading or for specifications grading, but with different skills that they’re building up. It’s also true for traditional grading, although it feels more familiar to students, but it still requires some scaffolding for them to understand what exactly is going on in the evaluation in this class.

Rebecca: Can you talk a little bit about some of your own experiences with ungrading and how you have done some of that scaffolding, what some of those early stages of scaffolding might look like, to understand the system.

David: So initially, I was having I think it was three check in meetings per semester, where I would meet with every single student in my class, like a 10- or 15-minute discussion, and we would talk about where they’re at. And that inevitably brings out students’ questions, and they don’t understand this part of they don’t know what that is about. And so that was one thing that I did, right? Those were the collaborative aspects of ungrading, and it was overwhelming, it was too much to meet with every single student that often. I still love being able to meet with students. And I use that as one of the tools in the toolbox. But nowadays, to sort of help build that up, I have a system where… this is actually done through homeworks for me… it’s like periodic homeworks and there’s some mathematical content. And there’s also some little additional thing where I ask a question, and early on, it might be something like, “Okay, what is a specific goal related to… and then I might have some of the math related content in the class or as a specific goal you’d like to work on related to this.” And that’s a fairly small thing. I ask students to say something along the lines of “I really want to work on improving my understanding of this one thing” and I can either respond to that as “Okay, great. Here’s the way I can suggest that you work on that,” or I can respond to it as… well, I don’t say it this way, but… “I can see that you’re having trouble elucidating the goal and so let’s talk about that” instead. And then another week or two down the line, there’s another maybe slightly higher-level task having to do with assessing themselves or thinking about their progress in the class. And so they’re scaffolded and I can, at each stage, see if a student is succeeding, in sort of the way that I’m looking for them to be thinking about their progress. And if not, then I can pull in a, “Hey, let’s actually meet and talk about this. And then I’m gonna give you a task to practice with that.” It’s really individualized, which can be a tough thing. But for me, it’s really fun to actually see students grow in that particular way, in addition to growing in the mathematical content.

Rebecca: So faculty that want to move to less traditional methods, what are some small steps a faculty could take to get started?

Robert: Well, I really appreciate that question. Because we don’t ever want to give the impression that you have to go all in on one particular grading system in order to like, be cool, or be a good professor or whatever, because everybody’s in a different position. Some people may not be in a position to ungrade, and we are never going to come out and say, “If you really care about students, you will do ungrading or you’ll do standards-based grading or whatever.” It’s like people care about students all over the place and just can’t or have mitigating circumstances. So I would say we laid out these four pillars: clear content standards, helpful feedback, marks that indicate progress, and reattempts without penalty. If a professor takes one of those pillars, and kind of drills into it on just a handful of assignments, that’s definitely progress in the right direction. So for example, you could take your next test you’re gonna give, experiment with letting students reattempt it, or reattempt different versions of it. So go one test, one class, allow reattempts without penalty, or give feedback that really gets to the point, not gives points, but gets to the point of what’s being done well and what isn’t being done well. Just pick one of these four pillars and just go with it. There are some small scale methods you might think about, let’s say, instead of grading your final exam, how about ungrading your final exam. Just say you need to lay out some standards for what constitutes successful work on your final exam and maybe that’s like 70% on the content. And then you got to write some reflective essays, and say, if you don’t meet expectations on the final exam, that’s a minus on your course grade, or something like that. So you can roll in aspects of alternative grading systems, whatever seems to resonate with you on small-scale individual assignments. And that’s really good progress. It’s really good data for you as the instructor too. You run these experiments, you ask students how it went, and then you just make notes and you iterate on that. It’s exactly the same thing that we asked students to do in these grading systems. You try something, you get feedback on it from a trusted third party, you iterate on it and try again.

David: I want to add one thing, because this is something that I always see happen when people first start using some kinds of alternative grading. Just for yourself, write a really quick list of what you think the major topics are in your class, aim for 10 to 20 things that are the important things to get out of the class, and then go pick some of your exams or quizzes, or whatever, and match them up. “Oh, on this one I was addressing these things and on this one, I was addressing those things.” And that can help you actually think about, “Oh, you know, I’m not really addressing this thing in my assessments at all, that I said was really important.” Or “Holy Moly, I have hit this one thing over and over and over all the assessments at the cost of not covering a bunch of others.” The first time you do this, even if you’re not showing students what the standards are, it can be amazing. “Wow, I’m not assessing what I say matters.” And that can adjust just how your assessments are focused.

John: Sounds like you’re advocating a backwards-design process where you start out with clearly defined objectives, and then you make sure that there’s alignment. And also, I think it might be important to make sure that students see the connection between those things, and that it’s a transparent process, which I know is something you’ve advocated in terms of your work on assessment.

David: Absolutely. And sharing your reasoning with students I think is very valuable, bringing something in from above and saying we shall now do it this way, because I said so, doesn’t work and you’re gonna get a lot of pushback, but talking with students about why you’re doing things and how it’s good for them, that makes a huge difference.

Robert: And don’t call it an experiment, okay. [LAUGHTER]

David: Yeah.

Robert: I’ve read some syllabi lately. It says, I’m going to be experimenting with this experimental new experimental grading system. And of course, you know what that is going to sound like to students… that I’m the subject of an experiment. What am I doing here? And so you’ve instantly killed your buy-in.

David: I’ll actually say, I don’t even give names to what I’m doing. So I am ungrading a class this semester, I have never used the word ungrading or any other name for it. I find that giving a name to things like that can help people sort of reduce it down to like a one dimensional idea, rather than engaging with what it actually is. I’m just “Oh, this is what we’re doing, and here’s why we’re doing it this way.”

Robert: Yeah, I have to share this story. When I started doing specs grading I did actually put we are using a thing called specifications grading on my syllabus. So, I got a call from our research office on our campus, they had gotten a call from the parent of one of my students complaining to them that I was experimenting on their child without informed consent, that I needed to give an informed consent form to my students before this happened. And I thought, well, that’s a weird flex from a parent and they call up the Office of Research Compliance [LAUGHTER] and complain about the Professor, but as you can just see even just the barest mention… Just tell students what you’re doing, and why you’re doing it. If I have to name it, nobody cares what the name is, it may not even have a name. And all this we mentioned before, that most of the people we interviewed for the book are not using any sort of canonical form of anything. It’s just like a little bit of this, a little bit of that, like cooking in your kitchen, you don’t use just one spice, you combine them, that’s where you get your unique flavor of what you’re doing.

John: Would it be fair to say that, given the title of your book and focusing on grading for growth, that one of the things you’re addressing is how to shift students from focusing on trying to maximize their grades to maximizing their learning?

Robert: Yeah, absolutely, yeah. I think a lot of frustration that I know I feel and I hear from other faculty is students tend to spend an order of magnitude more time talking about what they have to get on exam X to get grade Y than they talk about the actual content in the course. And we teach math, and I think some of the students who struggle the most with our math are doing like these amazing statistical calculations to try to figure out what the average has to work out to be. It’s like, “where was this in my actual math lesson?” I remember riding the bus home one day, from our downtown campus back to where we live, and I overheard this conversation and I said “Oh, this is this has got to come to an end.” [LAUGHTER] And absolutely, we want students to focus on their growth and I think students want to focus on their growth, too. I mean, students really, in the end, don’t want to think about all this stuff. They want to think about, “How am I growing as a human being? Is when I’m doing really meaningful? Where am I as a learner? Do I have any value in this vast educational system that I’m entering into?” And so if we can even just orient one small thing, like our grading system, towards convincing students that they have value, and are human beings in a stage of development, I think that’s worth it.

David: And that means we need to believe it, too. So I like to tell people who are thinking of using an alternative grading system, think about why you’re doing it, and what your students are going to get out of it. But definitely approach it as like, “Okay, I care about my students, and I want them to succeed.” And we’ll sometimes talk about incentives… grading systems set incentives in a class, and that can feel sort of like you’re trying to mess around with people and then incentivize you to do things a certain way. Traditional grading systems set incentives that are really kind of perverse towards learning. And if we can change those incentives, or just remove some of the ones that aren’t as good, that’s really a lot of what we’re aiming for here. And yeah, just respecting how people naturally want to learn anyway, that just allows them to do what they are as human beings going to do anyhow, or what they would naturally do.

Rebecca: So everyone wants to know, when can we have your book in our hands?

Robert: Well, it looks like on the Stylus website, it says July. So that’s what we’re going with for now. [LAUGHTER] I guess it’s sort of out of our hands at this point. We have one more round of copy edits to do. But it says July, you can preorder it now through the Stylus website and should be in your hands this summer.

John: My preorder has already been sent in. [LAUGHTER] So I’m waiting for a copy.

David: Awesome.

Robert: Awesome. Thank you.

Rebecca: And you mentioned earlier something about a blog. Do you want to tell us a little bit about that?

David: Yeah, if you’re interested in some of our ideas, go check out our blog, it’s gratingforgrowth.com …all spelled out. So, Robert and I started this sort of as a way to workshop some of our ideas for the book. So you can see proto-ideas. And we’ve also posted some examples of some of the case studies of some of these interviews that I have done with people using a lot of different alternative grading systems there. Every week, we post something new, or we have guest posters. There’s a really huge variety of ideas that show up on there.

Robert: And if any listener wants to contribute a guest post, we are accepting applications at this time… not really applications, just reach out. And we’re looking to hear. Now that the book is kind of close to coming out, we’ve had this discussion, you and I, David about where does the blog go from here once the book is out, and I feel like a great use of our time is to elevate the voices of other people. I mean, there are so many people out there doing great work, who are not education rockstars, they’re not on the lecture circuit or whatever. They’re just rank and file heads down people doing great work in the classroom. I love those people getting their ideas out there and seeing what they’re doing. So to me, that’s been a really rewarding experience.

David: So gradingforgrowth.com.

Robert: Right. It’s actually a substack. You can subscribe to it. It is free. It’s never going to cost anything and it comes every Monday. Except for taking planned breaks for holidays, we haven’t missed a Monday in two years.

David: Well, now you’ve cursed us, Robert. [LAUGHTER]

Robert: No, I put the pressure on me because then I got to step up. I think I’m next, actually. No, you’re next.

David: Oh, crap. [LAUGHTER]

Robert: Sorry, Dave.

John: We will put a link to that in our show notes.

Robert: Great.

John: Are there any other topics that you’d like to emphasize?

Robert: Well, I think I would just re emphasize simplicity. I mean, simplicity is the ultimate sophistication. I think Leonardo da Vinci said that. And so, when you’re designing a class simplify as much as humanly possible, and no more. But I think your students are going to be well served by less, honestly. I think we tried to do way too much in our classes, and we will be well served by this too. I mean, I have another blog that deals with productivity and time management in academia. And that’s the key thing. I mean, a lot of faculty struggle with overwork and burnout or just doing too much and there’s a lot of stuff you can say no to and I think keep things very, very simple and that can be a whole lot. podcast about how to simplify your life in higher education, especially applyied to course design and grading systems. You don’t want to make choices in July when you’re planning your fall courses that you end up regretting by October.

David: I will say that something that did come up earlier, briefly, but there’s not one right way to do it. You should not feel like you must go all out ungrading, you shouldn’t feel like you have to do specifications grading exactly the right way. Find something that works for you, look for models, pick and choose the things. If you look at something and say, “How could that possibly work?” then don’t do that. And yeah, we’re not judging you. Find something that works well and it’s going to be good beer students.

Robert: Yeah, and especially the last thing you just said, David is so important. All this stuff is predicated on actually communicating with your students. Don’t just sort of wait for the course evaluations. You have to really get in and talk, actually talk with your students on a regular basis and get feedback and see how things are going. And that’s one of the great strengths of all these four pillars sort of oriented approaches is that it really does get students talking with each other and with their faculty members. And I think that’s, maybe in the end, the best thing about all this.

Rebecca: Well, we always wrap up by asking what’s next?

Robert: Well, for me, I know I’ve been spending the last two years not only as a faculty member, but working in our president’s office under what’s called a presidential fellowship. And I’ve been coordinating large-scale institutional cross-institutional teaching and learning initiatives, mostly focused on active learning spaces. And so that’s been a particular focus of mine for a few years. That is wrapping up now. But we’re looking at maybe some interesting and exciting extensions of that idea where we’re going to try to surface and looking for what our faculty are already doing in terms of instructional innovation. There’s a lot of stuff out there, like we said earlier, but even in our own institution, a lot of faculty are doing really innovative and creative things and effective things but are kind of working under the radar. So the President and I, and those around us, are kind of thinking how can we look and find these faculty, get them together, elevate what they’re doing, and sort of make Instructional Innovation a normative practice at our university. So that’s that’s kind of what’s next for me is figuring out how to make that work on a practical level.

John: Sounds like a wonderful plan. I know, we’ve been talking about some ways of doing the same thing at SUNY-Oswego. And it’s a challenge.

Rebecca: It’s no small task. [LAUGHTER]

Robert: It is, it is.

David: Something that I am getting more and more interested in is what helps instructors be successful in any kind of educational innovation, not just alternative grading and changes in their pedagogy, anything like that. But especially when it comes to grading, there’s so many variations. And in so many different situations where instructors are working, what are the things that help them best succeed? And what are the things they do that help their students buy in and succeed most? And so looking at those sort of things from a more fine grain perspective, I think it’s going to be somewhere that I’m going next.

Rebecca: That sounds exciting.

John: It does. And I hope you’re both come back and talk about the success of these initiatives in future podcasts.

Robert: We’d very much like to.

David: Absolutely.

Rebecca: Well, thank you so much for joining us and sharing your stories. And I know that many people are looking forward to your book.

Robert: Thanks a lot for having us.

David: Yeah, it was great to be here. Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]

284. Learning That Matters

Many graduates describe their college experience as being transformative, changing how they view the world and their role in it. In this episode, Caralyn Zehnder, Karynne Kleine, Julia Metzker, and Cynthia Alby join us to explore the role that college faculty can play in creating transformative learning experiences.

Caralyn is a senior lecturer in biology at the University of Massachusetts-Amherst, Karynne is the former Dean of the Division of Education at Young Harris College, Julia is the Director of the Washington Center for Undergraduate Education at Evergreen St College, and Cynthia is a Professor of Education at Georgia College. They are the authors of Learning that Matters: A Field Guide to Course Design for Transformative Education.

Show Notes

  • Zehnder, C., Alby, C., Kleine, K., & Metzker, J. (2021). Learning that matters: A field guide to course design for transformative education. Myers Education Press.
  • Mezirow, J. (1991). Transformative Dimensions of Adult Learning. Jossey-Bass.
  • Selingo, J. J. (2013). College (un) bound: The future of higher education and what it means for students. Houghton Mifflin Harcourt.
  • Selingo, J. The Future Learners. Pearson.
  • Learning that Matters website
  • Learning that Matters: The Course Design Institute

Transcript

John: Many graduates describe their college experience as being transformative, changing how they view the world and their role in it. In this episode, we explore the role that college faculty can play in creating transformative learning experiences.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Caralyn Zehnder, Karynne Kleine, Julia Metzker, and Cynthia Alby. Caralyn is a senior lecturer in biology at the University of Massachusetts-Amherst, Karynne is the former Dean of the Division of Education at Young Harris College, Julia is the Director of the Washington Center for Undergraduate Education at Evergreen St College, and Cynthia is a Professor of Education at Georgia College. They are the authors of Learning that Matters: A Field Guide to Course Design for Transformative Education. Welcome Caralyn, Karynne, Julia, and Cynthia.

John: Today’s teas are:… Karynne, are you drinking tea?

Karynne: I am and I was joking yesterday that I would have to go to Starbucks and get mine because all I have is Lipton, and I did. And so I’m having some Earl Grey [LAUGHTER] in my Hawaii Cup.

Rebecca: …where I would really like to be during our impending snowstorm. [LAUGHTER]

John: Julia?

JULIA: No, actually I am drinking coffee out of my trusty thermos as I do every morning. [LAUGHTER]

Rebecca: …there’s always one ,Julia.

John: Well, at least one.. And Cynthia? [LAUGHTER]

Cynthia: I am drinking a Tazo tea called glazed lemon loaf.

John: I haven’t seen that one.

Rebecca: It smells really tasty. I’ve had it. The smell though, is what really gets it.

John: And Caralyn?

Caralyn: I have a handpicked hand dried sweet fern and sassafras tea that my 10 year old who’s now into wild foraging blended for me.

John: Wonderful.

Rebecca: Well, that’s amazing. Can I have I have 10-year old? [LAUGHTER] I have some Awake tea today, despite the fact that it’s two o’clock in the afternoon.

John: And I have Darjeeling tea today.

Rebecca: That’s a different choice for you, John.

John: It is. I was looking for things I haven’t had recently. So I picked that one.

Rebecca: Score one for you, fail for Rebecca. So we’ve invited you here today to discuss Learning that Matters. Can you talk a little bit about how this book came about?

Caralyn: Well, we, many years ago, were all faculty together at Georgia College. And it started with Julia and Cynthia started a group focused on course design. And it morphed into what became the innovative course building group. It was this grassroots… sort of bottom up… we wanted more support, and collaborative work towards teaching. And we all began working together through this and doing workshops. And so we decided to write the book that we wished we had had, when we had first started teaching. We wanted it to be based in theory, but really practical, have a lot of strategies, be really conversational, and be collaborative, and really encourage people to work together. Because we found that sometimes teaching could be so isolating that working together and talking with other people was just something that gave us so much support and we enjoyed, and we wanted that for others.

John: This book is designed to help faculty create transformative learning experiences. What constitutes a transformative learning experience?

Karynne: Well, for our book, we actually used a Mezirow’s theory and then work really from John Dewey. And our definition is about fundamental change that learners undergo, if it is a transformative education, whereby they see themselves and they see the world differently. I taught teachers and I would always tell them, the person you will be when you leave this program is not the person who you are now. So it involves a lot of reflection, whereby you have an experience, you process that experience, and then you make meaning of that. And that changes how you are viewing yourself and the world.

Rebecca: So reflection is a critical part of that practice.

Karynne: Absolutely. And that’s really what we get from Dewey is the importance of that for learning

Rebecca: So, you start a chapter with a pre-flection. Could you explain to our listeners, what this is and why you use this approach? And how we could use it in our classes?

Cynthia: Yes. So it’s one or more questions that we have at the very beginning of the chapter. And I feel like they are just gold. I thought that for a long time. I’ve always enjoyed having individuals do some thinking upfront really before we dive in. But then in a recent study, I’m going to say it was probably 2021, around there, students who took a practice test, who answered questions before learning the material, outperformed their peers who studied it more traditionally, by 49% on a follow-up test. So then I thought, well, heck, I think these pre-questions are even more valuable than I ever imagined. And when you think about why, it makes a lot of sense, because first of all, some pre-questions, some pre-flection, gets people in a good headspace and it’s got them thinking along with what it is you’re about to introduce. I think it stimulates anticipation, because now that you’ve answered some questions, you’re curious to see are the authors going to agree with me? Disagree? What’s going to happen? And I think it can highlight gaps in your knowledge that if you answered some questions previously, and then as you read, you might think, “Okay, well, yes, I said that in my pre-flection. And oh, yes, I said that, oh, but I didn’t think about that piece.” I think it kind of shines a light on those pieces that maybe you hadn’t thought of before. So I just really, really highly recommend that not only does it make good sense for a book like thispre-free questions. And when I have students reading something for homework, I always have some pre-questions that I asked them to answer before they even ever start reading.

John: One argument for it too, is that it helps activate prior knowledge, it gets students starting to make connections, recalling what they already know, and sets a frame for them to put new material into that framework and elaborate on what they already do know. It’s a wonderful strategy and I should do more of it myself. [LAUGHTER] I advocate that very often. And I don’t do it as much as I should.

Rebecca: I know one of the things that I’ve discovered in using some of those strategies is that sometimes a topic is familiar [LAUGHTER] and so familiar is different than knowing. And so sometimes doing an activity like that can help someone recognize that it’s something they’ve heard of before, but they don’t actually know that much about it.

Cynthia: Absolutely.

Caralyn: And especially if it’s something where you know that there’s going to be some misconceptions or things like where the topic, how it’s described, maybe outside of your discipline is not the same as how it is inside, or the terminology has specific meanings. And it’s so good for uncovering that and so much more powerful than me standing up in front of the room, just talking about it.

John: One of the anchor concepts in your book is the principle of teaching towards equity. What are some of the ways in which faculty can work towards creating a more equitable classroom environment?

JULIA: One thing I want to start off by saying is that sometimes, because the stakes around equity are so high, we get a little overwhelmed. And when we start to think about how do I teach towards equity, and one of my lifelong goals as a faculty developer is to demystify the concept of teaching for equity. And so I like to say, at its core, it’s the process of humanizing the learning environment. And so what I mean by that is just approaching each and every student as a unique human with their own story and understanding that the story that they bring into the classroom will impact and influence how they learn, when they learn, what they learn. And then I also want to say to these folks, because I was there, and this idea, like how am I going to make the world more equitable? It’s such a big job. But really, we all as human beings have the innate tools to do this, because we’re social beings that live in a social environment. And we have a lot of practice in all kinds of parts of our lives, learning how to create relationships, how to build communities, how to live in relationship with each other. But it can be challenging in teaching, because we’re working against some pretty powerful social forces that lead us to treat students in our classes as if they’re a monolith. In particular, there’s a powerful collective story about who goes to college and why. And many of us have unconsciously absorbed this story about who goes to college and why and it does not relate to reality. It doesn’t reflect the reality of who’s in our classes. So a big part of what we need to do is understand how to make visible the rich complexity of the stories of learners in our classroom. So my advice is to start with the things you know, which is, if this is something that’s new to you, the very first thing I would say is just make space and provide value for building relationships in your class. So by that, I mean like devote some time where you’re building relationships, where students are building relationships with one another, and put some value on that. So if the currency in your classroom is points, make some points that are associated with building relationships so you’re communicating that this is actually a highly valued part of the learning. And if you’ve already done that, then I should say the second step would be thinking about structure and transparency. So building structures that are clear and transparent for students. So the transparent syllabus and assignments are a great way to start with that, the idea of making what’s hidden, visible for students, and that helps us unpack those stories, because that collective story that many of us have absorbed is the students that are coming to college already know what it means to go to college and for many of our students, that’s not true. So helping make visible what’s hidden. And then the third thing I would say, which is like a thread throughout the whole book, which is grab a friend or some friends and sit down and have some conversations about it, get a book, read it together, but find some partners in crime in here to help you figure out how how you’re gonna teach towards equity and what it might mean for you to teach toward equity. So you can find some really firm grounding and footing for that.

Rebecca: One of the things I really like about how you’re describing teaching toward equity is that it’s a spectrum. And that it’s not equitable or not, but you’re teaching towards it, or you’re moving in that direction, or you’re pushing the needle there. And I think that’s a much more palatable approach than something that feels absolute. And we all know, it’s not actually absolute anyways.

Karynne: Yeah, I think that’s actually woven throughout the book. We really try to encourage folks to take the smallest step regarding anything. And then we also very much encourage collaboration. So find a friend to do this with somebody who’s like minded, and you’re never going to get there. So we’re not there. But this mindset that you’re moving in that direction is really helpful. And I think that’s, like I said, woven throughout the book.

Cynthia: I just think so often, when we think about equity, we think of it sometimes only in terms of content, like the authors I’m teaching, the scientists I’m including, and so forth. But we also like to think about equity in terms of the strategies, not just the what we teach, but the how we teach. And I think oftentimes, that’s an area of equity that people haven’t thought that much about.

Rebecca: Those are all really good points.

John: So one area where perhaps there might be some inequities is in terms of class discussions, because some students would like to talk all the time, other students are a bit more cautious, and sometimes even think about what they want to say before they say anything. What would you suggest to create a more equitable environment for discussions.

JULIA: I’ll jump in and say, my favorite for this, and first, I would say practice in very low risk [LAUGHTER] situations. First is the circle of voices. So this idea that you’re moving around in a circle, and everybody has a chance to speak uninterrupted, so that you’ve lowered the barrier to entry, and that you’re practicing this regularly, so that every student has a lived embodied experience of what it feels like to speak before you let go of those structures, then they’re much more likely to engage once they’ve had that kind of an experience. And then any kind of structured protocol where students are not spending their cognitive power, trying to think about how they’re gonna navigate the space, because it’s really clear how to navigate the space. So they can think about the ideas and do deep listening.

Karynne: Another that we all tend to use, is having community agreements. And we’ll probably talk more about that. But going through that experience with learners, and saying, “This is what we are committing to, and this is what we will abide by.” And that way, those for whom it’s just really, really difficult to speak in a large group won’t feel put upon to do that. If your community says we’re gonna encourage people, but we’re not going to require that or we’re going to ask people to be mindful of how much they are speaking, but we’re not going to close them off if they feel the need to say a second thing.

Caralyn: And I say that sometimes we think about discussions, and we just envision like, okay, we’re all sitting around a table having a classroom discussion, but opening it up, thinking about Universal Design for Learning, and that multiple ways for students to express themselves. So maybe it is an online forum, or maybe one is this synchronous or asynchronous, so that it’s not a, okay, you need to get up and speak in front of 20 people, but maybe you get some time to write. And here’s where the pre-flection questions can really help too, because having some time to think and write beforehand can make for such a richer discussion.

Rebecca: I think the pre-flection also offers that opportunity to transition into a space. You’ve been in this other place, or I was at lunch, or I had this thing, or I had this other conflict on my mind. But then here’s some time to get in this space of what it is that we’re talking about, which does allow people to focus more. So you also advocate for a strategy of “dilemma-issue-question.” Can you talk a little bit about what this is and how it’s a useful strategy?

Caralyn: The dilemmas issues questions, or DIQ approach is basically a framework or a model for putting the course content or the skills that you’re helping students master into a big framing question or a societal issue that students care about. Because we need to provide the “why” we need to provide the “here’s the purpose,” the reason for learning this. So if I’m teaching evolution by natural selection, rather than just diving into “here are the criteria,” maybe pose the question of “which species will be able to evolve in response to climate change?” because now we care about learning about what do we need to know to be able to answer that important question. It helps students connect. It’s an equitable practice because they’re bringing their own lived experiences. They can see where the knowledge and skills are useful, and they get to be creative and do creative thinking, critical thinking, and it’s so much more interesting and fun to teach. You can just take it in so many different ways. And we don’t have to look too far outside of our ivory towers to see big societal issues that we’re all going to be facing, especially many of our students. And if we want to have hope for those things getting solved, then I think providing students with that sort of training and modeling that in the classroom is just so important.

Karynne: Not just the importance of doing this, but really changing your mindset about what is important content in your class. We’ve done a lot of work with other faculty on the content doesn’t have to be these 9 million things that you’re going to be tested on at the end of your chemistry degree, but rather, this ability to think in the present and in the future and solve problems that really, really matter. Hence, learning that matters. I think that’s important to point out. So I think it’s a jazzy name that we’ve come up with, for a dilemma issue and question, DIQs. But I also think this mindset is just so important to develop.

John: And students, I think, would find it easier to learn about things that they care about, where they see the intrinsic value of what they’re working on.

Caralyn: Yeah, because that’s all of us. I feel like every other article in the Chronicle or Faculty Focus, it’s like,” Oh, do we have a student engagement crisis?” And it’s like, “okay, well, how do we engage people?” We engage them by having things that they’re interested in and passionate about, and find purpose in, and that’s where you can have projects where students, they’ll blow you away with what they’re doing, and how much work and time they’ll put into it because they care.

John: One of the issues that people have been complaining about for the last couple of years, since we move back to face-to-face instruction is what appears to be a lack of student motivation. So one way of addressing it is asking students to work on things that they find interesting, and that they can see the value of. Are there any other strategies to increase student engagement and motivation?

Cynthia: Well, I want to start by saying that I really think the decline is real. When I’ve been at national conferences and just talking to faculty from all over, it just seems like it’s what is on everyone’s mind. I have absolutely seen it. It’s interesting to think about why is there this decline? Some of the students I’ve talked to have said, it’s just really hard for them to pay attention for such long stretches of time, when they got used to only paying attention, maybe for short periods of time, I think some began to question the importance of learning at all, especially in high schools. There were often times where teachers were told if the students do anything at all, pass them. And so what message does that send to our students? But a couple of really interesting things I’ve heard from students recently. One student said to me, sometimes I don’t think you professors recognize that these cutesy assignments you give us aren’t really preparing us for the future. And so I feel like anything that helps students better face uncertainty, deal with authentic problems, as opposed to ones that we’ve kind of created in the classroom. Those make a really big difference. And then, of course, some of my graduate students told me this. They had been undergraduates when the pandemic hit, they said, during the pandemic, we learned to cheat, we learn to cheat well. They were just right up front about it. And these are excellent students. And now we’ve of course, got ChatGPT, which makes it even easier if you want to cheat. And that’s something I’ve been studying a lot. And through studying chatGPT, oddly where I came out, after weeks and weeks of study was that students valuing the learning Is everything. Students, valuing the learning is everything. It’s the answer. It’s the foundational answer. And so the learning must matter. And so I’ve been thinking a lot about what we know about intrinsic motivation, and what makes someone value learning. They value learning when they have more autonomy, how can we increase autonomy? They value learning more when they feel a sense of mastery over what they’re learning. They value learning more when they see the purpose. And often the relationship-rich type of classroom also makes them value learning more. So every once in a while, I think, would we have written a different book if we’d written it post pandemic? If we’ve written it post ChatGPT? And I think the answer is no, I think we would have written the same book, because everything in the book is geared toward that type of teaching and learning that is so focused on intrinsic motivation and engagement and relationship building and connecting to the world beyond the classroom. It’s almost like we saw this stuff coming. I don’t know. [LAUGHTER]

Rebecca: I had an interesting conversation with a colleague yesterday about a conversation she had had with some graduate students that talked about why the students were in graduate school. And they said, “Well, I kind of got cheated out of my undergrad. I didn’t get the undergrad experience because of the pandemic.” And so the motivations that we might assume, that are not necessarily real, of why someone’s in school in the first place, it was just kind of interesting to hear the perspective that they’re not here necessarily to get a particular kind of experience, they feel like they didn’t get. So finding a way to get them to value the learning is really important. And knowing they’re not here because they are motivated, because they’re so excited about a particular topic, which you might expect of a graduate student, I think is really an interesting insight to consider.

Caralyn: And I think it connects back to what Julia was saying, we need to know our students in order to understand: What are their motivations? Why are they here? …and we can’t just assume that they’re coming in with the same reasons we did. We need to take the time, build the time into our courses, to get to know students and have those relationships.

JULIA: I’d also add, if we’re really serious about making students the agents of their own education, we really need to look at the structures of how our institutions are set up, because they’re just so patronizing in every way. Like when students come, there’s so many ways in which they get messages about how they are not able to make decisions about what they can and cannot do that the institution, the professor, that they all know best, and that they need to fit themselves in the mold of that. And that mold is often defined by that story I was talking about earlier, that one story about who goes to college and why? And there’s a lot of unlearning we need to do in higher education to create institutions that actually center student agency.

Rebecca: So we know that institutional change isn’t fast, and requires a lot of people to push against the current structure to change the structure. And one of the ways we can do that is thinking about our own courses, a place that we do have control over. So can you talk a little bit about some of the strategies we can encourage faculty to adopt or practice in the spaces they do have control over, that would help us move into this transformative space and move towards equity.

JULIA: One of the ways in which we do this in the book is we do it in thinking about designing learning experiences from a liberatory framework. And I’ll back up and say backward design has been a really valuable tool in faculty development and teacher preparation, and really has helped change the way in which we think about how we teach. So instead of allowing a textbook or some other driving force, determine what the order is, and the pathway for teaching, we’ve thought backwards about what are the goals we want. One of the challenges with that, I think, leans into what I was just saying is that centers, the faculty member’s thinking very much. It’s a faculty centered thinking design process. And so something that we really tried to do in the book is think about how might you decenter the faculty member in that process some, so that you can bring in some of those student perspectives. And so we did this in a couple of ways. I won’t talk about all of them, because it would take a really long time, but one I want to mention is using design thinking as an approach to complicate that backward design process. So design thinking is an approach that we borrowed, not just us, but lots of folks in higher education now are borrowing from product and software design. And design thinking really starts with centering the user of the design. So if you were a product designer, you would start by trying to empathize with the user. So for example, if you are a toy designer, you’d want to observe children in play and engaging with toys to understand how they engage with toys. You might also want to dig into some research about child development in your target age group so that you could think about developing that toy to be appropriately developmental. And so we translate this in an activity in our book using an empathy map. And the way that we did this, which I think is quite powerful, is we built some composite student personas that tell different stories about students in college, but they’re based on data. To build these I use the institutional data from our institution. And also if you’re familiar with Jeffrey Selingo’s student segments, we use those as well to build, I think there are five of them, that tell different stories about students in college, their histories or herstories, and also their goals for being in college. And then the exercise asked the educators to center themselves in that narrative, and think about what kind of messages might that student be getting from their family, from the college, from the society at large? What kind of goals might they have for themselves, and really think deeply about these before you write your learning goals and decide what activities you’re going to do and set up your learning environment so that process of backward design can really be influenced by having a deeper understanding of the types of students that are actually in your classroom. I’ll just say with a caveat, these student personas were derived from our data and every institution is different. So it really helps to make your own. And from the concept of design thinking, the best approach is to have access to the actual users, which is not always practical in higher education. But another way you might do this is to interview students who’ve been through your class to think about a redesign, you might interview them to understand how they engage with the material. And this is a great way to use an assessment technique through an empathy map.

Karynne: Could I add a couple of things? One is about the process and why we liked this design process. And that is iterative. And the more I talk, the more I’ve realized, oh, everything is just iterative. And so I really liked that we get to embrace that and realize that, okay, it’ll be different next time, it may get closer to the mark if I do this. The other thing I was going to comment on is, and we’ve all done this as well… So it’s not always practical to design the course, but sometimes co-designing with students is really, really powerful. And we’ve tried to take advantage of that when we have that opportunity, just again to send that message, like it’s not about the professor’s experience, it’s about the learners’ experience.

JULIA: Even taking little pieces and co-designing them… I taught a general chemistry class for years and years. And I had a rubric for the final grade and we just co-designed that every year. But it was the only thing we co-designed because we didn’t have time to do the whole course. But that was a pretty powerful thing to co-design at the beginning.

Rebecca: As a designer, I appreciate everybody talking about design thinking. [LAUGHTER]

Caralyn: It took us a while to get there when we think about higher ed, but it makes so much sense. Who do we really want to be thinking about? …and it’s the learner and their experience.

Cynthia: And I often think about the who that we’re designing for, and that all too often novice professors, I find, tend to design for a younger version of themselves. Older professors tend to design for kind of an average student. And then every once in a while someone is designing for an anomaly, where they had a student a previous semester who did something terrible, and now they’re redesigning the course ao that never happens again. And I think any one of those can be problematic, and that we’re often better off trying to design with a variety of students in mind, and not just a single concept.

Rebecca: You mean, we don’t have just one student?

Cynthia: it turns out, we don’t. [LAUGHTER] ibut that would be nice.

Rebecca: It would be a lot easier.

JULIA:Getting specific is important here. The generalities are, I think, the problem. And so what the personas do is they provide some really specific cases to think towards. So you’re not thinking in general about a group of people that morph together, but you’ve got like, one of them is Juwan. And he’s a military veteran, and he can only go to school part time, and he needs to work two days a week. Just getting those details in your mind when you’re thinking about the design are really, really valuable.

John: Might it be helpful also, to get data from your specific students? Do a survey of them asking about their life experiences, about what has worked well for them in the past, and what challenges they’ve had in prior classes or where things didn’t work so well, so that you can address some of those in designing your course, perhaps co-designing, or at least responding to, the students expressed concerns.

Caralyn: There’s so much information there, and it helps going back to building those relationships, they want to be able to talk about who they are, especially if they see that you’re responding to their feedback and changing something because of it. That models such awesome behavior.

John: And if you know some of the things your students are interested in, you can use that sometimes to design activities that may appeal to the specific mix of students you have in your class. So you’re not teaching to that generic student, you’re working with the actual students in your class,

Cynthia: You could design even the assessments around those students sitting in front of you.

Rebecca: What? [LAUGHTER] Tell me more about that.

Caralyn: When we get into assessment, and this is where, when we were writing and that was this collaborative writing process, where I learned so much from Julia, Karynne, and Cynthia about this, and I feel like assessment is the area where there’s so much I can do, personally in my own courses, but also where I look at like that’s where we can have some of the biggest impact because I think our assessment practices have not been well designed and we have done harm and we need to fix that. And I think we advocate for connected assessment. So assessments where they are aligned with learning outcomes, of course, but also working and designing for the whole student. So they’re holistic, they’re affirming, so we’re not trying to be punitive. We’re not trying to like here, let’s go in looking for those mistakes. But we’re looking at, “Hey, where’s the growth happening? Where’s the learning?” …and highlighting that, and being so much more focused on giving feedback and process, so, “Here’s how you’re going and here’s how you move forward” and not just like, “Okay, here’s the percentage and you should know what to do with that,” because it turns out most of us don’t. And being able to have authentic experiences, and the end, like was mentioned earlier, being really transparent. Having examples, having models, being really clear about “here are the steps.” Because if we have, “okay, here’s a project, you’re going to write a lab report,” but I don’t describe actually what goes into that, and what are the steps in how to do it, well, then I really shouldn’t be surprised when the final products are not awesome, because I didn’t provide enough scaffolding to get there. And this is someplace where I’m still doing a lot of work here thinking about my values in teaching and how if I’m looking at that, now, for me, it might be that reading table on the syllabus, like here’s where the points are, here’s where things are coming from. How does that align with my values? How does that align with the message that I want to send students? And where we can being as intentional there as possible, and talking to students about what is the message they’re getting, because what I am intending might not be what is being communicated. And then where we can, really thinking about and being open to taking a risk with some alternative grading strategies. Maybe it’s ungrading, maybe it’s specifications grading, but there are so many more resources and great smart people doing so much work in this area. And every single one I’ve ever talked to or reached out to is always super excited and willing to share their ideas and share what worked and what didn’t, because it can just really change the entire feeling in a classroom when we take away the power of grades, because they’ve really been used to stop learning and oppress in many cases. And if we get rid of that, it really opens up the space for some honest relationships.

Cynthia: Unfortunately, you have to end a book at a certain point and publish it, it turns out, and one of the things that we didn’t really get to talk a lot in the book about was ungrading. We got more into it right after the book came out. But that’s where having a website that goes along with the book has been such a great help, because we were able to put so many fantastic resources about ungraving or minimizing grades on that website. And that made me feel a lot better. Because for myself personally, getting involved with ungrading has been one of the most important things I’ve ever done for my teaching. No one told me it was going to change everything. [LAUGHTER] I thought it was just going to change one little piece, but it changed everything.

Karynne: One of the things that I’ve tried to do with the ungrading is to share with learners… mine’s a view, it’s not the only view… and I never want to be punitive with grading. If you feel like I’m punishing you with grades, please, we need to talk so that I can know more about your assets, know more about your desires, and help you head in the right direction not punish you because you don’t know something. It’s like that’s what learning is. And so that’s just been a practice of mine.

John: We’ll share a link to the website for the book in the show notes so that people can explore some of the additional resources there. One of the things you advocate throughout the book is the use of active learning approaches. But you also note that you should probably expect some pushback from students. What are some of the most effective ways of addressing the pushback from students who prefer learning by being lectured at so they can sit there passively without having to actively think about the content?

Karynne: One strategy actually you can use is to be upfront about it. So students in this course before when I’ve used these things, some of them really don’t like it, they’re very uncomfortable. So I just want to tell you that I’m aware of that. And that’s actually a point where I bring in that idea that I don’t wish to be punitive regarding assessment, you’re going to have as much say in this as I do. So the other thing is to share the the literature and the research. And again, since I primarily teach people who are going to be teachers, they really need to know about what the literature has to say, what the research has to say about learning. And it occurs when there’s some space between absolute comfort and absolute chaos or uncertainty. There’s going to be some uncertainty, so we always try to share that with learners as well so that they can go back and tap into that research. Another thing that we really try to do is to use self assessment and reflection as much as possible, so that you’re letting us know where are the ways that you are growing. I may not be aware of all the things that are changing in you and if you are able to inform me of that, that’s a much more informative approach, then, okay, I’m going to do all of the assessment. We had to learn this ourselves, [LAUGHTER] to start expecting the push back. And then they think that you don’t like them, because you’re not teaching the way that they prefer. And emphasizing that collective, how we’re all changing, how we’re all growing here, I think is another approach that you can take.

JULIA: I would also add just tapping into their lived experience of learning something new. And often they can really embody that if it’s not something about school. So like, are you good at tennis? What did it feel like when you first picked up a racket? Or did you try and learn something and give up on it? Why was that? So understanding that actually getting really good at something does have this period of discomfort before it becomes a regular part of your life so that you understand that that is actually getting you to a point where you’re going to be a different person and transformed.

Cynthia: A friend of mine noted that her students at first they said their fear was that the assessments weren’t going to match the activities in class. And so that made me think, oh, that’s probably something I need to say right up front is, this is what the assessments are going to look like. Here’s how what we’re doing in class is going to feed into that, because I can see where there may have been professors they had in the past, who taught in a way that was very active, but then assessed in a way that was very passive, and students might have had trouble making the match.

Rebecca: Well, there’s been so much great insight in this conversation today. So thank you so much for that. We always wrap up by asking: hat’s next?

JULIA: Well, that’s an exciting question for us. And I actually want to start by talking a little bit about how we ended the book, because a thread in the book that we haven’t talked too much about is really focusing on the educators identity development as an educator being a really critical piece of this whole journey. So what you’re putting together for students and doing for students may often feel like it’s just all work that’s flowing out of you. But also a very important part of that is your own development over time. And so our last chapter is called “Your turn, self and collective efficacy,” and it was really important to us to end by saying to educators, it’s important to think about who you are as an educator, and invest in yourself that way. So that’s one thing that I just wanted to put out there and make sure that people understood that that was a value for us. And in terms of what’s next for us is we are really, really, really excited about launching a course design institute that’s based on the book, which we’re going to host in August, it’s August 4th to 7th, it’s called “Learning that Matters: the Course Design Institute.”Iit’ll be here in Olympia, Washington, a really lovely place to be in August. And we’re at Evergreen State College, which is a College in the Woods, very beautiful. We have a farm and a beach. And, [LAUGHTER] I know, we’re very lucky. But the idea is to have an immersive collaborative environment to design or redesign the courses that you’re going to teach in the next fall. And to do that with people who are not necessarily at your institution. So to get a variety of voices and feedbacks. We’ll have a lot of time for you to work on your own, but also a lot of time to talk with people from different kinds of institutions who are working on different kinds of problems, teaching different kinds of courses, to build that interdisciplinary approach to the work that you are doing in your classroom and also help you build a wider community. So this is something we’re super, super excited about. And we will share that link with you so you can put it in the show notes. And then as a little bit of a teaser, we’re doing a free virtual workshop on May 9, this is at nine o’clock Pacific Time, which is noon Eastern time. It’s called Making Courses Memorable Beginning and Ending and I’m not going to say more about it because I want your curiosity be sparked there.

Cynthia: And of course, we’re also always happy to zoom in with people who are using the book for book clubs.

Rebecca: Well, thank you so much for joining us. We look forward to sharing this and encouraging folks to pick up your book.

Caralyn: Thank you.

JULIA: Thank you.

Karynne: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

281. The New Science of Learning

Students who enter college without a preparation in effective learning strategies often do not persist to degree completion. In this episode, Todd Zakrajsek joins us to discuss what incoming students should know to successfully navigate the college experience.

Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of 4 Lilly Conferences On Evidence-Based Teaching and Learning. Todd is the author of many superb books. His most recent book is the 3rd edition of The New Science of Learning: How to Learn in Harmony With Your Brain.

Show Notes

Transcript

John: Students who enter college without a preparation in effective learning strategies often do not persist to degree completion. In this episode, we discuss what incoming students should know to successfully navigate the college experience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Todd Zakrajsek. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of 4 Lilly Conferences On Evidence-Based Teaching and Learning. Todd is the author of many superb books. His most recent book is the 3rd edition of The New Science of Learning: How to Learn in Harmony With Your Brain. Welcome back, Todd.

Todd: Well, thank you so much. I’m looking forward to our conversation today.

Rebecca: Today’s teas are:… Todd, I hear you have a surprise for us.

Todd: Yeah, actually, I’ve got a bag of mystery tea. There’s just a whole bunch of different teas in here and they’re little packets. So live in an air we shall open up one of the packets.

Rebecca: So, would you like a drumroll? [LAUGHTER]

Todd: There we go. And now I am going to be having crystal clarity oolong tea to find a peaceful state of mind. Nice.

Rebecca: Sounds like a good state of mind to be in.

John: So it’s a fermented tea.

Todd: Apparently it is.

John: Where’s that from?

Todd: This is from Portland, Oregon.

John: Excellent. And I have a blueberry green tea from the Republic of Tea.

Rebecca: And John, just because you were asking about it last time we recorded, I have my last cup of Hunan jig, just for you.

John: Very good.

Rebecca: I do not know why it’s called Hunan jig. [LAUGHTER] It’s tasty.

John: We’ve invited you here today to discuss the third edition of The New Science of Learning.

Rebecca: Can you give us a little overview of the book?

Todd: So the book is essentially a guide to help faculty and students to understand the learning process, but also just the whole college experience. Now, this is not a book that’s like tips on how to study, specifically, it’s more of a global looking at teaching and learning. It does have tips in there too, and actually each chapter has a couple, but that’s not the foundation of what I’m really after. For instance, the first chapter is about learning from multiple perspectives. And it talks about the dangers of dichotomous thinking. Too much in our society, it’s either I like it, I don’t like it, that’s a good person, bad person, and gets away from that. And from there, there’s sections in there on setting goals and self regulation, monitoring how we interact with others in our work and self efficacy, the extent to which we believe we can succeed at something. There are whole sections on helping to understand how people learn, finding patterns, and what that does in our society, in our classes, and in our content. If we can find the patterns, we can learn a lot more easily… Bloom’s taxonomy, and it has a chapter in there on sleeping, the effects of not sleeping or how much it can help you when you do sleep and exercise. And it even has a chapter in there about how to work well in a group. So it’s essentially kind of an overall book that helps students with the learning process or the college experience.

John: The book is clearly a good resource for first-year students, that said books often have more than one audience. Did you write this book for a broader audience or was it focused primarily on first-year students?

Todd: It’s always tricky writing a book, in my mind, at least you have to have your audience in mind the whole time you’re writing, that’s the only way I can do it, and write at a level that I think connects with the audience. So certainly, first-year students and more specifically, like a first-generation college student, or a student from a marginalized group that doesn’t have a lot of experience in their family with colleges. Because if you do, it’s a very different experience than if you don’t. So this is a resource for people who don’t know the ins and outs. At the same time, there’s a lot of material in here that faculty just don’t know. And so some of the learning theories that are in here, some of the pattern recognition, some of the sleep research, the faculty don’t know. So I tried to write it so that faculty would also find it interesting. And I tried to straddle that line, but I also tried to pull in what a senior in high school might find valuable. So a junior or senior in high school could read this and get a better sense of what they were going to experience in college. So I tried to do that, and then a general resource for anybody else in terms of people in student affairs or in a student success center. So I was looking at multiple audiences started primarily with the student. But when I used examples and the level of writing, I tried to drift in and out so that I could get these other groups in such a way that they would find it valuable as well.

Rebecca: I really enjoy the personalized conversational tone, which obviously is great for students. It hooks you right in and then goes into the introduction. And so I really enjoyed that style. Can you talk a little bit more about why you chose that style and how that might help students?

Todd: For me, I just think conversation storytelling is one of the most effective ways of learning and so I like to do that. I also like to bury things just a very little bit at the beginning. So you start to read and then you realize as its unfolding… So, for instance, in the first chapter, we’re talking about the danger of dichotomous thinking. The example in the book was: it’s easy to tell night from day. If it’s noon, we know it’s day and if it’s midnight, we know it’s night. But what happens just as the sun sets? There is a moment when the day stops and the night starts and it’s the edges like that where all the richness is. And then I think there’s a line in the book that says, once we’re going through that, like the day and the night and what’s really at the edge, and it’s like, we’re not really talking about day and night here, are we? We’re talking about people. And so that kind of concept that I really like in terms of keeping it conversational, keeping it tied to things that people know. But my whole goal, and what I’m shooting for is to help people who can read science, but do it in a way that they enjoy it.

John: Much of your book focuses on how we learn. Students come in with some serious misperceptions about how we learn. When students are asked how they study, they tend to read things repeatedly, where the evidence suggests that’s not very effective. They tend to highlight quite a bit, which is also not very effective in increasing long term-recall, or transfer ability. Why aren’t students learning how to learn before they’re, say 18 or 19 years old or older? Shouldn’t some of this instruction be taking place in earlier years of education? And why isn’t this happening earlier?

Rebecca: John, did you bring your soapbox with you today? [LAUGHTER]

Todd: It’s an issue. I mean, it’s a huge issue. We spend our whole time in our educational system teaching people stuff, how to do things and what things are. But it’s crazy, we don’t teach students how to learn. We treat it as if it’s an implicit assumption that everybody just can learn and to an extent we can. When we’re young, we learn how to walk and we learn how to use utensils, and we teach kids how to tie shoes, and children learn how to ride a bike. And so I think in the general framework, there’s all this learning going on and teaching going on. And as a result, we have the implicit assumption that everybody can teach and everybody can learn. But teaching is a profession. And learning is really, really nuanced in a lot of different ways. And so what we have are a lot of implicit assumptions and trials and errors. If you think about for yourself, where did you actually learn how to learn? And for most individuals, it’s around second or third grade, because that’s when we start testing. Which by the way, if you ask young, young children: “Do you like to learn and do you like school?” they say yes to both of those, until suddenly, they start to say that they like to learn and they don’t like school. And it’s almost universally in the country around third grade. So right around third grade, we’re starting to test, but we don’t teach them how to learn at that moment. So the parents are making up flashcards and quizzing the kids and the kids are reading aloud in class. And we’re going through these actions without knowing what we’re doing. And it turns out, as you’ve already pointed out, John, very well is that a lot of these things that we have implicit assumptions about, we’re wrong, we’re just wrong. And the trouble is, we don’t have a baseline. So if we start highlighting, if I highlight the chapter in the book and I get a good grade, then obviously highlighting must work. And if I’m underlining things, and I get a decent grade, underlining must work. But how much could you have learned if you realized how to do that? Now I’ve got students who will use five different colors to highlight. When I ask them why the different colors, they’ll say, the blue is if it’s an application, and the green is for vocabulary, and the pink is something that’s like really important, and I always love to tell them when they do that. You should use black for the stuff that’s not important at all. Good times. But the idea is they’re doing that the students sometimes are doing their flashcards but the question becomes… like flashcards, when you’re going through a deck of flashcards, when you get it right, do you set it off to the side? Or do you put it back in the deck? How do you do that when you’re learning something like chemistry? How do you learn those terms? When you’re learning a periodic chart, how do you do that? And so I just firmly believe if we started teaching children how to learn at around second or third grade and just spent 1% of the time teaching the learning part and the rest of it all about content, by the time that they were done with school, they would be lifelong effective learners. And instead, we have people who believe that they have a given learning style, which we could do on a whole different show. People do have different ways in which they learn, but the concept of teaching to a given learning style has no data behind it. And highlighting, there’s studies out there that says highlighting doesn’t work and it doesn’t work primarily, however, there are ways to highlight that are effective. Re-reading is not effective, unless you reread for a specific purpose or reread in a special way. And so this stuff is going on, and, again, we’re not teaching students how to learn. So we should do that. I’ve given presentations about how to learn to everyone from high school students up through professional schools, in nursing programs, pharmacy programs, and medical schools. And the number of times that someone in a medical school… a second-year medical student… will come up and say I wish somebody had taught me this sooner. The case you’d think of is like the prototype of a student would be a med student who memorizes and learns stuff so fast. But those students who can pick up things quickly will say I wish somebody had showed me how to pick it up even faster. And so I think we should do that. Same with writing. We should teach students how to write instead of just having them write. There’s a soapbox for you. [LAUGHTER]

Rebecca: I mean, I’m on it, too. [LAUGHTER] It’s interesting that when I was thinking about your question and about like, when would I say that I started learning was actually when I started struggling.

Todd: Oh, interesting.

Rebecca: …because when it wasn’t hard, you could just skate by, but there was a moment, and it was in sixth grade, I remember, social studies, and I had a really hard time reading and reading comprehension. And then I had someone who actually had to read more effectively. And it worked immensely. But it was only because I had that intervention or that help. Because it wasn’t part of the curriculum, it wasn’t taught. that I actually overcame that. But I think a lot of our students come to learning these strategies once they’re struggling significantly to the point where they have to ask for help, rather than us being proactive about it.

Todd: Exactly. And I tell you, and it’s in the book here, too, is it’s exactly what you said was my experience. I pick stuff up very, very quickly, I could skim a book and go in and take a test and do well. And I did that through high school. When I got to college, I had five classes. My first class that I got a grade back in was like a D minus in the introduction to criminal justice class. And then I had a physics class and my first grade and that was an F. And I thought, well, what’s not going well. And then in my math class, I got an F minus. And I remember thinking, well, it can’t get any worse than this, until I got my chemistry grade and that was an F minus minus, I even went to the teacher and said “F minus minus? I don’t understand this.” And he said something like, “Given you received an F minus minus, it doesn’t surprise me you failed to comprehend it.” So, a kind of mean person too. And the concept here, and the reason I mentioned is what you just said, Rebecca, I hadn’t learned how to learn. And so at the point where I needed to know how to learn, I was in a jam. And I actually went to the registrar to get a drop slip. And she said, “Get your signatures and bring it back. I’ll take care of it.” It was a long time ago, and four of my five faculty members signed the slip. This was a very small school, this isn’t some big school where you get lost. This school only had like 200 faculty members and about 3500 students and the psych prof said, “I don’t understand what you’re doing. Why would you drop out?” This was like two months after I started. And I said, “I just can’t do it. I don’t know how to do it.” He said: “You need to learn how to learn.” I said “like you can learn how to learn.” I didn’t even know the concept existed. And so he pointed out some strategies and pointed toward a book and I learned how to learn but I was one signature away from not finishing, I wouldn’t have met you, I wouldn’t have done any of the books I’ve done all of that would have not happened for one signature, because nobody taught me how to learn.

John: And a lot of our students get those last signatures and disappear. We’re losing a lot of students once they hit that barrier, which is why it’s important we have books such as yours, and we spend more time working on teaching students how to learn.

Rebecca: And reading the book as part of our system.

Todd: That’s what we should do. If I’m doing a faculty workshop at a campus, and I say “How many of you in here came within a whisper of flunking out of school?” most faculty raise their hands. And that’s just amazing to me, those are the folks that you would think got through easy. So it’s what you just said, John, how many fabulous, wonderful people, they’re probably doing things that are fine, but they’re not doing what they wanted to do and it’s because of that.

Rebecca: Yeah, we just need to design our systems to be proactive, rather than reactive. And oftentimes, it’s not even reactive, we just miss the boat entirely.

Todd: That’s a good point. Instead of being reactive, we should be either proactive, or at least not inactive.

Rebecca: Let’s start with not inactive, yeah. [LAUGHTER]

Todd: That’s a place to start.

John: Part of the issue is that we ultimately figured it out on our own. And we assume that everyone can and we’re not a random group of the population, a very large share of faculty members were not first-generation students. A disproportionately large number of faculty members come from families where there were people with higher ed is part of their background. And it’s easy to forget what sort of struggles students may face. Even if someone may have come close at one point, they figure it was an aberration, and they forget that those aberrations can be critical points for many people.

Rebecca: And that these struggles happen across the spectrum. It’s not just our undergraduate students. As you mentioned, our graduate students have some of the same struggles. I was just having a conversation with graduate students last week about even just basic time management skills or how to troubleshoot or problem solve, because they don’t have those skills, and they need to build those skills.

Todd: Yeah. And it’s still also not equitable across different groups, individuals from marginalized groups tend to fail more frequently, because they don’t have the resources and they don’t have that support system so that when they are struggling, somebody can help them.

Rebecca: So the first edition of this book was released in 2013. How does this third edition differ from these earlier editions?

Todd: Actually, in a lot of ways. When we wrote the book in 2013, first of all, the research has changed considerably. But the book ended up also being just a hair over 100 pages. And this new version is about 250-260 pages. So it has grown substantially. There’s sections of the book that were not in the original book, or not even in the second edition. So there was a whole section on how to learn in groups. There’s a section in there on pitfalls, the places where students tend to have problems. Hidden curriculum kind of issues: what are things that they’re not specifically stated and so they’re implied in a way that if you know that they exist or you had family members who went to college you can figure it out. But if you’ve never gone to college, you didn’t know, I didn’t know when I went to college that if you failed a class, you could retake the course later. And so I thought when I failed my chemistry class that I was literally done, because if you can’t pass, then I can’t get into Chem II. If I can’t get into Chem II, I can’t go… And when I talked to my advisor, the advisor says: “Well, just take a trailer course.” And I said, “What is this thing you call a trailer course?” So those types of things are in this edition of the book. So I picked up a lot more nuances than we had before. And of course, I mentioned a little bit earlier, too, but the research has changed significantly in the last 10 years. We know a lot more now about how we learned than we did 10 years ago. And for things as subtle as what’s happening while you sleep. And so it’s getting more and more that we know actually, what kinds of learning is being solidified at different stages of sleep. So there’s always changing research, and I’m just happy to be able to get that updated research in there.

Rebecca: I love that you just slipped in something about sleep, because I was just going to ask about sleep, I was just having a conversation with a colleague today about being able to process new information when you’re tired. And that we might typically think of processing being associated with a learning disability or something. But actually, lack of sleep can cause the same kinds of symptoms, essentially. And so I can imagine that actually talking about sleep as an easy sell for students, because it’s something that everyone can easily think about, but many of them don’t get. Can you share a little bit of insight into sleep and learning?

Todd: Certainly, and this is one of those areas that we all know that it’s harder when you’re exhausted to do something than when you’re rested. But back to the dichotomous thinking, we think oftentimes in terms of I’m exhausted and I’m rested. But what about all those nuances in between. What if you normally like to get seven and a half, eight hours of sleep, and you get six and a half hours. You feel okay, but what we know now from the way people learn is that you’re still going to be learning at a less effective level. And if you’re exhausted, you get to a point very easily where you can’t learn at all. And so we know that in terms of encoding the information, you need to be able to process the information in your environment, which happens when you get sleep. So that’s important. And we know that nobody wakes up after a terrible night of sleep and says, “Whew, I feel great, I look great, and I’m learning like crazy.” We know it’s going to be a rough day. And so that fatigue makes it hard to learn. And then what we also know about sleep, which is fascinating to me is while you sleep, a lot of consolidation happens, it’s called consolidation. And if it doesn’t happen, the information is gone, typically in about 24 or 48 hours. And so what we have are students who, for instance, will study all night and they can go in and take the test. They do okay on the test, so they think this is an okay way to learn. And then they don’t realize that the material is pretty well gone in two days… three max. And then later when they need the material, if let’s just say for the comprehensive final, the instructor says “Oh, everybody, I hope you’re studying because this final is going to be tough.” Now I go to learn for the final… I flunk the final, most students don’t say “Oh, I’ll bet that’s because when I learned I didn’t get stage four sleep, which consolidated the information and therefore made it available for me to relearn it at a faster pace for the final.”No, they come back with a “Wow, that was a really hard final.” So it’s going on all the time. But sleep is probably right up there at the top of one of the things you can do to learn more effectively is to sleep well.

Rebecca: Yet, so many of our students don’t sleep. And we inevitably are probably teaching a class full of students who haven’t had a lot of sleep.[LAUGHTER]

Todd: Yeah, and it’s for the public service announcement, we got to put it out there because the sleep is important in terms of learning. But there are so many things that are tied to lack of sleep, it’s just incredible: diabetes, even cancer, weight gain, high blood pressure, all these things. There’s just tons of stuff. Your skin actually looks worse. There’s so many things that are tied to a good night of sleep. It’s when all the restorative stuff happens. So I’m going to tell you listeners, the folks who say, “Yeah, I know, I’m exhausted, I can’t get my sleep.” It’s damaging to a person to not get sleep. And when somebody says “Well, yeah, but I got so much to do,” just keep in mind that it will take a toll. And oftentimes, and this is an important one, if you get a rest or get some extra sleep, you’ll do other things so much more effectively, that you come out ahead and don’t have the health issues.

John: And this is really important to convey to students. And I do share this information with students in my classes. I don’t always practice it myself, unfortunately. But I do share the information. And when they see results on how much more they recall when they’re well rested, at least a claim it will have a bit of an effect on them in the future. But one of the things… this is more on the faculty side rather than the student side… but so many of our classes are designed in such a way so that faculty are using high-stakes exams. Students have a lot of incentive to cram the night before a test and it does have that immediate payoff of increasing their short-term recall. And then, since they’re worried about the grade, they don’t necessarily care about how much they recall until they get to their next high-stakes activity. And then they have to go through the whole process again. And maybe this is something that faculty should work on too in terms of reducing the number of high stakes activities, reducing the incentives for students to cram and to cut back on their sleep.

Todd: Yeah, I think that’s a really good point. In fact, there’s several things that we can do to impact the student’s sleep. When I mention the importance of sleep to faculty at times, they’ll say, “Well, I can’t make them sleep.” And oftentimes, my response is “No, but you can keep them up.” If you have a high-stakes exam, and it’s like a midterm and a final, it’s human behavior, people are going to wait toward the end to do it. I know, there’s some faculty out there listening who say “I do everything early.” And that’s great. But I can tell you, I’ve been on a lot of committees with my colleagues, where we turned reports in at the very last minute or somebody handed me their portion at the last minute. So it is going to happen, if we know students are going to wait toward the last minute to do it. It’s what you just said John, it’s a good point, if it’s a huge exam, it means they’re going to be up, maybe even for multiple nights. If it’s a big paper, they’re probably going to spend the night all night writing it, maybe two days and it might get a little bit of sleep, but they’re going to be tired. If you have the paper due on like Monday at noon, they’ve now got exhausted from Sunday night, they’re gonna be tired all week. If you could make your paper due on Friday afternoon at like 2:00, if they stay up all Thursday night, now they’re exhausted, but they’re exhausted going into a weekend. So a lot of little things we can do. I have a friend Howard Aldrich at UNC, he had a nine o’clock class, 9 am. He had the papers due at class time in the morning, then he and I were chatting and with Sakai, you can see what times papers are turned in. So we were looking and the students were turning in the papers, two o’clock, three o’clock in the morning, even eight o’clock in the morning. And he knew that they stayed up to do it. So he changed his deadline to 9pm with a 12-hour extension, if you asked for it. So if you can get a 12 hour extension from 9pm to 9am. What he found was 86% of his students turned in the work by 9pm. So when that happens, we can’t say they were awake. But we know that they weren’t up doing his paper in the middle of the night. And so those are the kinds of things that we can do as faculty members. And I agree in terms of the high-stakes tests, we can think through what are we doing that’s actually going to be conducive to learning versus is going to make a hindrance. And if we say “Well, it’s their own fault. They shouldn’t wait till the last minute.” Why put them in that position?

John: To be fair, though, to those faculty who do give high-stakes exams, they often spend a lot of the time just lecturing in a monotone, which can facilitate sleep on the part of students, at least during their class time, which is a large share of the time that they’re interacting with students.

Todd: That’s great. Yeah, I suppose they could… get a little nap in during class. that could work.

Rebecca: It’s all about balance. [LAUGHTER]

Todd: It is balance, isn’t it? That’s a work-sleep balance right there.

Rebecca: I had an interesting conversation with students this week about perfectionism and procrastination, which also, I think leads to sleep deprivation because of all the procrastination. And what I found in the conversation… students were being really authentic and open with me… was that they were so worried about their performance on things, even low-stakes things, like these weren’t big-stakes kinds of things, but just so worried about their performance on something that they would wait to do it. But they’d spend all this energy and time worrying about it. And so we talked about how to actually take an assignment and then plan it and break it into smaller pieces. But I talked to the students about how to break it into smaller pieces so that there were times to get help, because they were so worried about not doing it well that they could build in time to get assistance and help. So I’ll be interested to find out at the end of this week, if they were going to try this strategy this week to see if it helped them. But it had never occurred to them to break it into these smaller pieces.

Todd: Yeah, and what you just said, I think, is vital for anybody who’s listening. It’s all the stuff that never occurs to somebody. This is why individuals who go to therapists can gain so much is when a therapist had some an individual says “I never thought of it like that.” For students, let’s look at your sleep. Just jot it down on a scale of one to 10 in the morning, how did you feel about how much sleep you got? To what extent did you get a good night’s sleep, and then at the end of the day, jot down on a scale of one to 10, how’d things go. And when they start to see that bad night’s of sleep result in days that are not all that productive or work well, it’s like, “Oh, I didn’t know it was that related.” Breaking things into tasks, I think is fabulous. That’s what this book is about, too, is the concept of just showing them things and then having them be able to look at and say, “Oh, I had never thought of that.” And that’s what’s valuable. So I like what you’re doing.

John: For those who procrastinate on coming up with the set of tasks to do, again, course design could resolve that a little bit by scaffolding the project so that students never have a huge chunk of work to do all at once.

Todd: Yeah, I think that’s good. And then the other one is a whole different program is ungrading. And if we can just remove some of the grading on some of these things, and there’s faculty out there and myself included years ago who would say well, “If I don’t grade it, if there’s no grade, why would the students do it?” And it turns out, sometimes, from what you just said, if the students are so stressed about it, they spend all this extra time, you remove some of the high-stakes aspects of it, and they don’t stress about it so much. But that is a problem. And I will tell you, it’s not just the students, I was writing a blog, and I have a person I turn a blog over to McKenzie. She’s phenomenal. And she edits at the end. And I wanted to prove to her I was working on this one blog, because I told her I was going to get it and and I kept getting busy with other things. And I submitted it to her. But I planned on spending another three or four hours on it. She emailed me back, and she said, “This is so close to being done. Let me just edit it. And then you can take another look at it.” Had she not said that I would have worked another probably four hours on this thing, half a day. And I think students are doing that at times too. I think they finish an assignment. It’s good. And then they think, “but I want it to be better.” And so, just clarity and helping to understand and building some structure into the course so they’re not guessing. Take away the stuff that’s just not necessary and let them focus their energy on the things that are necessary.

Rebecca: Sometimes it means even pointing out something that’s low stakes is actually low stakes.

Todd: Yeah, I think that’s really good. Signposting. So there’s a terminology for you. Signposting is basically telling somebody what they’re doing, or what you’re doing. So if I’m giving you feedback, I could just give you feedback. And we’ve had programs where at the end of the program, the students will say, “I’m not getting enough feedback.” And so we all as faculty say, all right, anytime we give feedback, we’re going to say, “Would you mind if I give you some feedback right now? Hey, would it be alright, if I give you just a little bit of feedback? Do you have some time tomorrow for some feedback?” …and at the end of the semester, the response was too much feedback. We hadn’t changed. But it’s what you just said, just let people know what’s going on.

John: One of the nice things about your book is that it’s grounded in learning science, but it’s really easy to read. One of the things we had trouble with in coming up with questions is there’s so many things that we could discuss in this book that we thought we’d shift it back to you, what are two or three pieces of advice that you would recommend to students that might have the biggest impact on their learning?

Todd: First of all, I appreciate the fact that you found it easy to read. And I have gotten that feedback from others too. It’s called “The Science of Learning.” And I think that scares some people at times. This is not a dry book, I tried to make it conversational, and folks say it’s fairly easy to get through. And that’s good. The couple things… we’ve already talked about sleep quite a bit. Sleep is just huge if we can help talk to students to sleep. The other one you had mentioned already is the cramming. The tricky spot with cramming is not necessarily that the students want to do it, they are reinforced for it. I consider this to be one of the biggest traps in higher education. Because the research suggests that if I cram all night long, don’t sleep, study all night long, and if you sleep for six or seven hours, I may very well outscore you by two or three percentage points, just enough that I do fine, and it looks like that’s okay. And you’ve already mentioned that a couple of days later, and the information is gone. The students don’t realize… sometimes they know it’s gone later. But they don’t generally know that it’s going to go away at the extent that it does. What they know is they’ve studied, they did well on the test, and therefore they’re doing okay in the class. So, a couple things in the book, if we could help them understand how much damage comes with cramming, it would be huge. In fact, it’s in the book like five times, to the point where the editor said, “Do you know you’ve already talked about this like four times?” And I said, “Yep, with any luck, we’ll only do it once more. [LAUGHTER] But it’s that important.” So that’s a big one. The other thing that I think is really huge is if we could help students with understanding metacognition, the concept here is knowing when you know or understanding your learning process, and it’s something that we don’t monitor, but we could. When you sit down to study, jot down how long you think it’s going to take you to read the chapter, when you’re done reading the chapter, jot down how you felt it went, jot down a couple of notes of what you learned. As you’re reading, stop every couple of paragraphs and just look away from the book and think, “What am I reading right now?” Because your mind will start to wander and you not realize it. Everybody that I know has read a chapter or read an article and either the next morning didn’t remember if they had read it or not or even when they finish they thought to themselves well, I don’t remember anything about that. I was thinking about bacon the entire time. And so that concept of just knowing when you’re processing… so metacognition is big, the sleeping stuff and cramming is big, and the last thing I’d say… there are lots of things in there… but just understanding Bloom’s Taxonomy, understanding at what level you know something. I like to use this as a quick example, I’m from Michigan, you could teach your students that there are five Great Lakes. Imagine they know only this, there are five Great Lakes: Huron, Ontario, Michigan, Erie, Superior… HOMES, right? And we could say Superior is the deepest. I can come back with a quiz two days later and say which of these lakes is the biggest Ontario, Michigan Erie, or Superior? And the students could say Superior. At this moment, I don’t know for sure if the students know what a lake is. I asked them these are five things called lakes. This one’s the deepest. Later I say of these things called lakes, which is the deepest?t They’ve memorized it. If students know at that moment, they’re just functioning at the recall level, ot helps them and it’s because when they take tests they start to understand I’m doing well on recall and understanding, I’m not doing well on application. So knowing Bloom’s, knowing metacognition, understanding the sleep thing, and then exercise is huge. There’s all kinds of research out there that says, if you’re actually getting your heart rate up 15-20 minutes a day, it does all kinds of cool things for your brain and actually makes learning easier. So that’s just a couple of them.

Rebecca: So in the description of your book indicate that there’s an instructor’s manual that accompanies the text. And often this is not the case. [LAUGHTER] So can you talk a little bit about what’s included in the manual?

Todd: Yeah, so when I was writing the book, the first and second editions didn’t have this. And other books of this ilk don’t tend to have it… the first-year common reads, and the first-year experience books… but I wrote an instructor’s manual when I was early career faculty member and I wrote it for an introductory psychology book. All textbooks have instructor’s manuals now, so I thought, why should this book not be just as good as those. So when I was done, I kept right on writing. And I’ve written an instructor’s manual, which, ironically, is about as long as the first edition of this book. So what I did is for each chapter, I understand that if you’re going to use this book in your classes, you may not have time to read things very, very carefully, you might have to skim a chapter at times. So each chapter has a summary. So in the instructor’s manual, that summarizes the major concepts in the chapter, every chapter has discussion questions at the end. So I put down these are the types of things that students may very well say in the discussion questions. So that if you started discussion, you’re not stuck with a situation of asking the students to discuss, you show up in class and you think, Ummm,I’m not sure what I would say about this. So I’ve given you a couple things. There’s also teaching tips in every chapter. And for each one of the teaching tips, I’ve got a short thing of these are the kinds of things that students should experience. And on top of that, every chapter also has active learning exercises. I’m big on the active learning, So it will say in the sleep chapter, here’s like four different things you can do. And it sets it all up, it explains: here’s what you tell students to do, here’s what you have them do, here’s how you report out. And so it’s kind of a guide for active learning keyed to the book. And you can find this, if you go to the Stylus Publishing website… you’re actually not going to see it unfortunately, if you go to Amazon, because that’s not where it’s listed… you have to get to the Stylus publishing site, and then you can find it and there is no charge for it, you just let them know that you’re teaching a course and they’ll send it to you. And if you can’t find it, I’m the only Todd Zakrajsek in the world. So if you send me an email at ToddZakrajsek@gmail.com, then I will make sure that I’ll get you connected to the person with the instructor’s manual because we didn’t make it real easy to find, because we didn’t necessarily think that the students should have the instructor’s manual. [LAUGHTER] So it’s kind of buried in there a bit.

John: And we’ll include a link to your email in the show notes.

Todd: Perfect.

Rebecca: So can you share one of the examples of an active learning activity that you might do in relationship to the book?

Todd: Oh, sure. The chapter on sleep, there’s one activity that’s kind of explained there for keeping a sleep activity log for a week. And it shows how to have students block off their time and then indicate whether or not things went well, or it didn’t go well for them. And it helps them to find their ideal time. So I did this when I was an undergraduate. And it was fascinating, because I found out that between 2 and 4pm, I’m practically worthless, but early early in the morning, like it’s 6 to 8am, if I do have to get up and do something, I was just really, really good. And I don’t care for getting up early in the morning, so it was unfortunate, but that’s what I found out. Another activity, and there it’s called a snowball technique. And this particular one in the chapter on sleep was students are asked to think about things that help and hinder a good night of sleep for them. And then the snowball aspect of it is they talk to other students, and then they learn one thing that helps and one thing that hinders sleep, and after you learn from five different people, you go back and sit down, you get into a small group, and then you discuss those, and then you report out kind of overall, what are the general themes that you saw. So there are things like that in the instructor’s manual, they’re described in like a half a page. So it doesn’t take you very long to read through it and get a sense of what it looks like. And so it’s there just to help you get you rolling.

Rebecca: Sounds like it really reduces some cognitive load for faculty teaching these things.

Todd: One of the issues that is tricky that we do have to be careful of is faculty are really, really busy. And I taught one time on a quarter system, so you had three quarters, and I was on a 5-5-5 load with a total of nine new preps. So there were times that I was really struggling in running into class last minute, and then I had multiple sections and everything. And it would have been really helpful to have a 750 word that I could read in five minutes summary of the chapter, so then I could talk to the students. Because I knew the content. I just had to make sure I knew what was in there. And then for an activity sometimes drawing up an activity is not easy. If I could glance at one and get a sense of it, then I can do it. Same with the discussion questions. And so yeah, busy folks, and it’s just to help them out when they get in a bit of a jam.

John: That can be extremely helpful especially with those sorts of teaching loads, which I’ve only experienced once or twice, but it’s really challenging.

Todd: You do what you do.

Rebecca: …sounds terrifying. [LAUGHTER]

Todd: It is not easy. It’s not.

John: In the last section of the book, “A Message from Dr. Z,” you note several struggles you had in pursuing your own education. Why finish the book this way?

Todd: Well, I’m really glad you asked that. Because the last section, students oftentimes won’t read the things that are way at the end, I think the faculty are the same. And I probably have been the same too. But at the beginning of the book, I talked a little bit about some of the struggles that I had when I went to college, to almost flunking out and the fact that if one faculty member had decided to sign the form, I wouldn’t have been writing this book. And so that’s where it started. But I saved the message for the end, so there wasn’t the end of the story. And so when you go back to the “Message from Dr. Z,” that section starts with “Welcome to the end of the book,” And it’s essentially, “Let me tell you the rest of the story.” And what I did from this, there’s a great quote by E. McClellan basically says… it’s attributed to him, as there’s a lot of variations. But it boils down to everyone’s fighting a battle we know nothing about, everyone’s fighting a hard battle, it’s worded different ways. But that’s been really impactful for me, because I think if everybody knew that everybody else was fighting a battle at any given moment, then we could have a little bit more patience with individuals. But we also could say, you know, they’re getting through it, maybe I can get through it, too. So I finished the book with just a real strong narrative, in a sense that when I went through school as a first-generation college student, I almost flunked out that first semester. If you almost flunk out the first semester, just keep moving forward. And I had to work a lot of jobs, I was exhausted, but I had no money. And so I was working all the time. And so if you’re working all the time, you’re going to be tired, just keep moving forward. And I had a daughter when I was a graduate student and graduate school is already hard enough until you have a child and I almost quit graduate school because it was so hard to have a child and work in graduate school. So if that happens to you, keep going forward. And I almost ran out of money multiple times. And I would have dropped out. One time, I probably stayed in school because of $100. But I do actually know a couple of programs now, there are a couple of institutions that will give up to $500 to a student. You just show up and say, “Look, I really need $200.” And they find the students don’t abuse the system, but you don’t want someone to flunk out for $200. But in my case, at the end of this book it’s like if you’re struggling with some money, just keep that in mind. And so I just want to tell you real quickly, I don’t like to usually read these things, but just to give you the tone for the end. So I put this in there, “I leave you with the following to consider in the months ahead. Be mindful of your past, but look to the future. Listen carefully to the voices of others and find respectful ways for your own voice to be heard. Find ways to get what you worked so hard for without taking anything away from anyone else. Most importantly, always strive for more so that you have more to share. Ever forward. So that’s the tone I want to leave the students with. We’re all struggling at times and it’s not going to be easy, but if you just keep moving forward, we can make it.”

Rebecca: Speaking of moving forward. You’ve been doing a lot of writing, five books in the last five years. Are there more books coming? Are you going to take a break, like what is going on?

Todd: So I have ADHD, which means I have spent my entire life with too many things just kind of banging around in my head. So it turns out that once I started really writing, I got on a roll. I didn’t write much in my career, and it’s funny, I haven’t. And when I got rolling with some of the things, I’ve had so much fun. And so yeah, the five books in the last five years, I have another book that probably will be done in the next couple of months. And that one’s on helping new faculty to get rolling. And then I have another book that’s already signed. And that’s dealing with more with a kind of a longitudinal, how we learn and kind of walking through the learning process in a different way, which is cool. And I’ve gotta tell you, I’ve known y’all for a while there. But I’m really thinking that I need to write a book that I’m so excited about. It’s basically Dr. Z’s crazy stories, [LAUGHTER] stuff that I have kind of gone through in my life, and it’s what I’ve learned from it. So I had a student who had a grand mal seizure in my class one time, I have had all kinds of issues, lots of things have happened. And I think that there’s some stories in there that I could kind of tell because I do love telling stories. And it would help faculty, if I say, here’s how I handled this thing, and here’s what I faced. My goal would be almost the same as the end for the letters to Dr. Z for students, it’d be for faculty members of some crazy crazy stuff is gonna happen to you. And there’s ways of getting through that

Rebecca: Dr. Z’s case studies.

Todd: Yeah, that would be fun, wouldn’t it?

John: You might want to make that unreadable, though, by grad students until they’ve already started their careers, because otherwise some people might decide to back away.

Todd: No, no, no, John, they’re gonna find out that we get through with these things. And there’s also some really fun stuff that happens too, so that’s all good. You know… Alright. Maybe we don’t show it to grad students. That’s a good point. I’ve tried to defend a position. I was trying to defend an indefensible.

John: Grad students are already struggling often, so it may be best to wait until they’ve at least started.

Todd: Yeah, that’s a good point.

Rebecca: New title: Dr. Z’s survival tales. [LAUGHTER]

Todd: Oh, I’ve got to cite you on that one. [LAUGHTER] That’s good. That’s probably a better way to go with that.

Rebecca: So we always wrap up by asking: What’s next?

Todd: The book I just mentioned… what’s next. And I’m actually looking forward to getting back on the road. So kind of the what’s next is because of COVID and everything I hadn’t gone and done many workshops at campuses. And I’d love to do that. I’ve been on like 300 campuses. So it’s just fun visiting places. So I do have a couple places that I’m heading to. I was just at Anchorage, Alaska and that was really fun. And you find yourself in Anchorage, Alaska, and a couple days later, you’re in Florida. So it’s kind of an interesting thing. But I love looking at different places and traveling. So it’s been great. So the next is, I’m getting a chance to travel again.

Rebecca: So you’re getting close on your 50 states?

Todd: So, we have talked about this in the past. I’m gonna come back with my pleading of the listeners once again. I have been stuck at 49 states for about eight years. I hit my 49th state, I think it was seven or eight years ago, which I believe was Vermont… it was Vermont or New Hampshire, the order was really close. But North Dakota, it continues to be elusive.

John: North Dakota and Montana were the last two states in which an episode of our podcast were downloaded six or seven years ago when we first got started.

Todd: It’s interesting.

John: I don’t think there’s too many colleges there.

Todd: No, they’re not many colleges there. But I’m glad you did ask that. Rebecca, this is crazy. Because again, if Tim Sawyer had signed that form, I never let that go… is because anyone listening right now you never know when you’re the person who could say, “You know what, I’m going to choose something different than just letting you go.” It’s a big responsibility. But there’s times when a single sentence will change a student’s life. And so I can’t believe it when he said that, but I have now been invited to and presented in 49 states, 12 countries and four continents. Just amazing. And I would feel better if North Dakota would just call me, that would be so nice.

John: The last continent might be tough.

Todd: Yeah, there’s one continent that’s really tricky to get a gig in.

Rebecca: It could happen.

Todd: It’d be helpful if there are people who live there. [LAUGHTER]

John: The penguins are not that impressed.

Rebecca: You could just invite yourself. [LAUGHTER]

Todd: See that was the rule, by the way.

Rebecca: Oh, that’s right. That’s right.

Todd: Yeah, the rule was you had to be invited, so you can’t just show up someplace and start talking.

Rebecca: Well, we look forward to keeping tabs on your 49 states. Next time we talk to you. It’s always a pleasure, Todd.

Todd: Thank you… appreciate the opportunity to come in and I’llI get the 50th state,I will give you a call and maybe we can do a show about my 50th state. [LAUGHTER]

John: And I think sometime before that might be nice as well.

Todd: Wow. There’s a little pessimism for you.

Rebecca: Geez. [LAUGHTER]

John: Well, that may take a year or two.

Todd: Gosh John… Rebecca. Wooh. That’s tough. That’s brutal. Alright.

Rebecca: I was thinking that was gonna come soon, Todd. Sorry.

Todd: This is a mic drop time. Why don’t you go ahead and do the outro at this point. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

265. The New College Classroom

Despite all that we have learned from cognitive science about how people learn, the most common form of classroom instruction still involves students passively listening to a lecturer standing at a podium at the front of the room. In this episode, Cathy Davidson and Christina Katopodis join us to discuss alternative approaches that treat student diversity as an asset and allow all students to be actively engaged in their own learning.

Cathy is a Distinguished Professor at the CUNY Graduate Center, the author of more than twenty books, and a regular contributor to the Washington Post and the Chronicle of Higher Education. She has served on the National Council of Humanities and delivered a keynote address at the Nobel Forum on the Future of Education. Christina is a Postdoctoral Research Associate in the Transformative Learning in the Humanities Initiative at CUNY and has authored over a dozen articles on innovative pedagogy, innovative pedagogy, environmental studies, and Early American Literature.  She has received the Dewey Digital Teaching Award and the Diana Colbert Initiative Teaching Prize.

Show Notes

Transcript

John: Despite all that we have learned from cognitive science about how people learn, the most common form of classroom instruction still involves students passively listening to a lecturer standing at a podium at the front of the room. In this episode, we explore alternative approaches that treat student diversity as an asset and allow all students to be actively engaged in their own learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Cathy Davidson and Christina Katopodis. Cathy is a Distinguished Professor at the CUNY Graduate Center, the author of more than twenty books, and a regular contributor to the Washington Post and the Chronicle of Higher Education. She has served on the National Council of Humanities and delivered a keynote address at the Nobel Forum on the Future of Education. Christina is a Postdoctoral Research Associate in the Transformative Learning in the Humanities Initiative at CUNY and has authored over a dozen articles on innovative pedagogy, innovative pedagogy, environmental studies, and Early American Literature. She has received the Dewey Digital Teaching Award and the Diana Colbert Initiative Teaching Prize. Welcome, Cathy and Christina.

Cathy: Great to be here. Thank you for having us.

Christina: Thank you so much.

Rebecca: Today’s teas are:… Cathy, are you drinking tea?

Cathy: I am drinking English breakfast tea.

Rebecca: Yes. [LAUGHTER] And Christina?

Christina: I am drinking English breakfast tea.

John: And I am drinking wild blueberry black tea.

Rebecca: I forgot what kind of tea I made this morning. [LAUGHTER] I have no idea. I made nice loose leaf tea this morning, and it’s tasty, but I don’t remember what it is.

John: So you’re drinking a tasty tea.

Rebecca: I’m drinking a tasty tea this morning.

Cathy: Will I be kicked off the show if I say I’m drinking coffee?

John: About a third of our guests do. Yeah, and sometimes water.

Christina: I also have a Diet Coke. [LAUGHTER]

Rebecca: Sneaky, very sneaky.

John: And I had that right before I came over here.

Christina: We’re drinking all of the things.

John: We’ve invited you here today to discuss The New College Classroom. Could you tell us a little bit about how this book project came about?

Cathy: I can begin. This is actually the third in a series I called the “how we know” trilogy, which I began in 2011 after I stepped down as Vice Provost for Interdisciplinary Studies at Duke University, my previous employer, and was really interested in the science of attention and the neuroscience of learning. So that book really is about the neuroscience of learning. Then I wrote a second book called The New Education which came out in 2017, and was re-issued in a post pandemic version just this past spring. And it’s called The New Education. It’s really the history of higher education and why we inherited the forms we have now and how much higher education was re-created for and rebuilt, redesigned explicitly for the industrial age. And then I wanted to do a kind of installation guide that actually showed people how we do these things, how you take our knowledge of how the brain works, take our knowledge of learning, and how you take our knowledge of history and why we’ve inherited this very cumbersome history and actually do something new. And I thought it would be ridiculous for someone at the end of a long career to be telling other people how to teach. And I had the great fortune to be working with this… Christina will close her ears right now, she gets very embarrassed when I say this… but this utterly brilliant scholar, an Americanist environmental scholar, sound studies scholar who also had written several essays on pedagogy and had won all the teaching awards. And I asked her if she’d be interested in co-writing a book with me. And we started writing it when I was a senior fellow to the Mellon Foundation in my beautiful office overlooking the courtyard of the Mellon Foundation. And then the pandemic hit. We made a pledge to one another that we’d meet every Tuesday and Thursday and write together and we literally wrote, rewrote, re-re-re-wrote, and then re-re-re-re-re-wrote [LAUGHTER] every word together during the pandemic. So when people say, “Can you have a real relationship? Can you have a real project during the pandemic?,” we would say “Absolutely.” And that’s actually kind of key to the book, ‘cause we talked about learning in all its facets online and face-to-face. But that’s the very long version of how I’m the luckiest author in the world to have been able to work with Christina.

Christina: I’m the luckiest. [LAUGHTER]

Rebecca: It must have been very nice to have the stability of consistent writing times with each other during a time that was so unstable.

Christina: Yeah, it was a lifeline. I live in Brooklyn, near the Barclays Center where a lot of the Black Lives Matter protesting was happening here. And kids will be playing in the street. They had to close the streets on the weekends and weekdays so that kids could get time outside and neighbors could gather together outside where it would be safe. And kids playing on the street, drawing all these beautiful things in sidewalk chalk, and then you’d see police completely decked out, batons out, pepper spray, all kinds of things, getting ready for the protest that was just completely peaceful protesting and a police force that was just prepared for something wildly different. And it was a really dark time in trying to protest and think of a better way, a more equitable institution, an institution that could prepare students for the world, to be citizens of the world and also to fight for social justice and racial justice, and getting together and writing this book. It felt we have even more purpose more so than ever, in writing this and leaning into the active learning methods that really prepare students to participate and engage with the world.

John: You begin the book with something which actually serves as a very nice introduction to much of the rest of the book, which is a story about a 50-person department meeting in which no one was responding to the department chair. Could you just share a little bit about this anecdote?

Christina: Sure. So I was in that department meeting, it was a meeting in which we were given a really big task of imagining the goals for the department for the next 10 years. This is a review that happens every 10 years. And the department chair was standing at the front of the room behind the podium and was like, “Alright, so what are our goals for the next 10 years?” [LAUGHTER] …and everyone was quiet. Because as you can imagine, everyone has things that they want to change, things that they think could be better, and no one wanted to be the first to speak. And it really just felt like one of those situations where there’s a bunch of dry straw, and someone could just light a match. And so I was like, “Okay,” I knew everyone there, and I was familiar with the chair. The chair was very supportive of students, and I knew he was willing to listen. And then I said, “Hey, can we just talk to someone next to us and come up with a few ideas first, before we speak with the whole group?” He’s like, “Sure.” And this is what’s called think-pair-share where everyone thinks about a question and then they pair up with someone next to them or a small group of people. And then when we’re all done, we come back and share what we came up with. And five minutes go by and he is trying to get the attention of everyone in the room, the room has exploded in all this conversation, but everyone is smiling and enjoying talking to the person next to them. They’re thinking more hopeful thoughts with their generous colleagues and students and faculty all together. And when the chair was finally able to call everyone back to order… I’ve done this in 8 ams… the poor people who teach next to me have said that my classes are too loud at 8 am. [LAUGHTER] And Cathy has done this with a lot of people every time she gives a talk, and it takes a long time to get everyone to come back to order. And then people were so eager to share what they had talked about in their groups. And there was a little bit of anonymity, because it was like, “Okay, everyone in my group said this, not just me,” [LAUGHTER] and everyone was willing to share and we started to envision some really beautiful goals for the next 10 years that were really hopeful, that were imaginative, and creative, and beautiful, rather than starting with critique. Or sometimes what happens is, if one person says something, then everyone else kind of jumps on that train, and this way, diverse number of ideas coming out of these different separate conversations. So that’s why we do it.

Cathy: If you want an education-ese term this is called an inventory method. It’s the opposite of the standard seminar where you ask a question and those same three students raise their hand dutifully and answer the question. And sociologists of education have studied who those students are who raise their hand. And they tend to be a good match for the professor in class and race and gender and family background, family income… the people who are most into the class, most likely to get an A plus, most likely to go on to graduate school, most likely to be professors, replicate their professor. And that’s one reason why only 1% of Americans have a PhD and 25% of the professoriate has a parent that has a PhD. We have a system that’s a closed system. When you do an inventory method, like the one that Christina uses, think-pair-share, everybody in the classroom contributes. In that case, it was a meeting. But that’s true in a classroom too. Sociologists of education also tell us that 20% of students graduate from college without ever having spoken in a class unless they were required to speak by a professor. But if we really believe that higher education is about empowering students, not just giving them content, but giving them the tools to be experts themselves, then they have to learn how to articulate those ideas. And of course, some people are shy and then having them write on an index card means they’re still participating , even if they’re too shy to actually say something in class. An incredible method that I learned from a second grade teacher and I’ve done in many situations. Christina was alluding to the famous one I tell all the time about trying think-pair-share with 6000 International Baccalaureate teachers in the Philadelphia Seventy-Sixers auditorium. So they’re all thinking and pairing and sharing on these jumbotrons in the auditorium.[LAUGHTER] It was great. And I’ve also done it with the top 100 performing CEOs of the Cisco foundation. I did this with the Board of Trustees at Duke and John Chambers, the CEO of Cisco was on, he said, “I need you to talk to my executives, because there’s nobody more likely to be reticent about giving comments than one of my CEOs who’s talking to me. Power does that. There’s a kind of silence that happens in the face of power, whether it’s with second graders or a department meeting like Christina was in or students every day in our classes.

Rebecca: I think what’s really interesting about both of the stories that you’re sharing is that we’ve been in those situations where the silence is overwhelming, we’ve maybe have been the person running the meeting or the classroom or also been the person in the audience.

Cathy: And it feels awful, it feels terrible. My students call it’s playing silence chicken. [LAUGHTER] That’s a great term.

Rebecca: It’s a perfect descriptor for sure. So given that we’ve all had these experiences, and also have experienced the opposite. I think probably all of us, or at least most of us have experienced those engaging opportunities. Why do we always default to the one that doesn’t work? [LAUGHTER]

Cathy: Every structure we have in academe tells us our job is to learn from the master. And I use that word in quotation marks, but pointedly, and then repeat back what you learn on a final exam and that’s how you get As. And the students in our classrooms, they’re the winners, not the losers. They got to college because they learned that lesson, Freire calls it the “banking model,” where it’s my head dumping and depositing stuff into your head. We also have studies that go back to the 1880s, not 1980s, 1880s, the Ebbinghaus experiments with memory that tell us we forget 75% of what we’ve learned for that exam within weeks after the exam is over. So it’s not an effective way to learn, but it’s the way we learn. So we’re being reinforcing in a way of learning we only use in higher education and formal education, we don’t learn new skills that way, when it’s not commonsensical to learn from a lecture. I don’t learn how to play tennis from a video. I might look at a video, but that’s not how I get better. I practice and I improve and someone corrects me, and then I change what I’m doing. We all know that. But, for education, we’re told, learn from the sage on the stage. That’s what you’re supposed to do. That’s how you excel and our students have learned how to excel. It makes complete sense that they would think they learned more from a lecture than when they contribute. They’re deferring to the authority in the room.

Christina: And I think that this has been ingrained in them for a long time. We’ve been ranked and rated since birth, literally the first minute of birth ranked against others. And this one wonderful researcher, Susan Engel. She’s at Williams College, developmental psychologist, and she cites a study where kids on average before they go to school, when they’re at home asked about 27 questions per hour. And anyone who has a toddler, sometimes it is well more than 27 questions per hour. [LAUGHTER] My toddler just started asking why about literally absolutely everything. He’s about three. So right on target for asking lots of questions, for being naturally curious about the world. She talks about this. It’s called epistemic curiosity. And Dr. Engel says that once kids go to school, they ask on average three questions per hour, which is just a precipitous drop in getting to ask your questions, getting to know the world around you through engaging with it. And so by the time they get to college, they expect to learn more from a lecture, that they are not experts in the room, and that the person at the front of the room has all of the answers. And they have been trained through standardized testing to believe that there is a right answer, and they just need to be able to get to that right answer. That doesn’t train them to take on the world’s toughest problems and be problem solvers, to find alternative solutions, to think outside the box. And maybe those problems in the world are caused by believing there is a right answer and not thinking that maybe it’s me sitting in the room with all of these other people sitting in the room who could find a better solution or a better answer than the one that that one person has. And that’s how we change society. That’s how we transform our institutions. And I think they’ve just been ingrained in this system for so long, that by flipping it around and saying no, there are 50 different ways to answer this one question or 50 different ways to solve this one mathematical equation, then it’s more up to them. It’s giving them a little more responsibility and autonomy so that they can practice using it when they get out into the world rather than thinking, “Oh, there’s someone smarter than me who clearly has that figured out and they’ll take care of that problem.” That doesn’t really give them the kind of responsibility and accountability that I think that we all need in the world.

Rebecca: What you were talking Christina, I was thinking about a conversation I had with my kindergartner over the weekend, who does still ask many, many questions, some of which I do not know the answer to and so I don’t remember what she was asking me but whatever it was, I did not know the answer. And I said, “I don’t know, we’ll have to look that up” and she’s like, “I’ll ask my teacher, she knows the answer.” She’s in kindergarten, she’s only been in kindergarten for two months.

Cathy: I interviewed kindergarten teachers and first-grade teachers about learning. And almost every kindergarten teacher said, “when kids come to them, they think they can do all these things like ‘I love math, I love music, I love art, I’m an artist,’ by first grade, within six months into the first grade, they know, ‘I’m not good at art, I’m not good at math, I’m okay at language, but I’m very poor at…’ and already have absorbed those kinds of lessons about themselves.” And what you said is great, because it’s like the authority of the teacher is already happening. And then what happens when you absorb those lessons into a self definition. And that’s what we’re working against with active learning, is not just definition of the teacher, but definition of yourself in your role as a person.

John: And that’s hard to correct at the college level, because students have been indoctrinated in this from their very first exposure to educational systems. And when faculty do try using active learning, they often get a lot of pushback from the students. And that’s a challenge for new faculty where their teaching evaluations may have some impact on their continued employment. So it’s a difficult cycle to break.

Cathy: Yes, and it’s one reason why we include the research, serious research with any thing we offer to faculty members about what they can do in their class. And we also talk about the 2014 meta study in the publication of the National Academy of Science that looked at every possible way of evaluating learning and said if this had been a pharmaceutical follow-up study that Eric Mazur, who’s one of the inventors of the flipped classroom, did at MIT because his brilliant MIT students, were all sure they were being shortchanged by active learning. So he had them read serious scientific studies of active learning and then they all thought they were doing great by having active learning, and they thought they were better. So it’s about using the methods that speak to people in order to change the methods because unless you address the actual present situation of the audience, of the students, of the people you’re addressing, you can’t change things, you have to honor that present situation before you can move to something else and make a structural change beyond that.

Rebecca: A lot of the current system of higher education, as you mentioned, is based on a really different era, a really different audience of students and our student populations have changed, become more diverse, there’s more people going to college now than before. So how do we help students who have been through this system that has not really invited them to the table to really get involved. We share some of the research, and what are some other ways we can support them on this endeavor, and to continue helping us change the system.

Christina: Some really great pedagogy out there. One I’m thinking is an assignment that Erin Glass does with her students where they read terms and conditions for all of the technology that they’re using on their campuses. And they have to closely read them and critique them to help them become more aware of capitalist surveillance and what they are required to sign up for. And then they write a critique of that, of the university and of the system that they’re being signed up for, so they’re not only learning more, they’re learning digital literacy. They’re also learning more about the institution that is guiding these things that they’re signing up for. And they’re becoming better critical readers in general. And they’re also talking back to that institution and saying you could be doing better. And so there are ways in which we can give our students real-world problems that are immediately close to them and to their experiences of education, and task them with coming up with something better. And it’s really wonderful that students who come from all different kinds of educational systems to get together and think of what could be done better. Or Cathy also does this where at the end of the semester, a final project could be like, come up with a better syllabus. I don’t know if you want to speak to that Cathy, but I love this assignment.

Cathy: Yeah, I love to end the class where I be in the next class, like I say, “Okay, we’ve had about 12 weeks or 16, depending on the institution that I’m at, together and we’ve done this as a syllabus, and you’ve contributed to the syllabus, the last assignment, and sometimes we make this even as a final exam, is to make a syllabus the next people who take this class will inherit. Put your stamp on it. What did you like? What didn’t you like? That’s an incredible activity and it means it’s but again, using an education-ese term, it’s metacognition, too, because it means students are looking back over everything they’ve learned, which is the best way to beat the Ebbinghaus 75% forgetting because you’re actually processing it, analyzing it, and then trying to come up with some new version that you’re bequeathing to somebody else. It’s a legacy that you’re bequeathing to somebody else. It’s a marvelous exercise and you can do it in any kind of class. It’s quite fascinating to see the different ways people can come up with things. So often people say, “Well, you’re an English teacher, so of course, you’re flexible.” But what about… there’s somebody named Howie Hua, who’s a professor at Cal State Fresno, who on Twitter almost every day he comes up with what he calls mental math problems. And he’ll ask something seemingly simple that you think there’s no other way to solve that, like, “Add in your head 24 plus 36? How did you do it?” And dozens of people respond with different ways to add whatever those two numbers are that I just said, in their head. And it’s fascinating, because the point he’s making is it’s not about the right answer. It’s about, not only understanding the processes, but understanding all the different tools that we can have in order to be better mathematicians, more passionate about science or more passionate about all kinds of learning. So I think any assignment that gives students the tools and allows them, and even better is when they can pass on those tools to somebody else. When Erin Glass does that assignment, they not only critique, but they come up with their own terms of service. What are the terms of service for our class? What’s the community constitution for our class? What are the community rules we’re going to form in this class? …and so they’re already invested in a new kind of structure even before you start populating what that structure is going to look like.

Christina: And I think from the first day, you can really set students up for something different. One thing that Bettina Love does in her classes is she has everything set up and she goes, “Okay, now you tell me when the deadlines are going to be for this.” And so students look at their calendars, look at their schedules, and determine when things are going to be due so that they’re not all due in the same week as midterms or as in the same week as finals. That’s a very student-centered approach, asking students to come up with the learning outcomes of a class. And even in the most restrictive situations, you can add learning goals and learning outcomes, even if yours are set by the department. So I think also asking students what needs to change to better serve them and centering them in that conversation, you can start with your class and then start to think more broadly about a department or a whole institution.

Cathy: What I love about all of these different things, and we’ve borrowed them from other people, one of the things we did was we interviewed so many people, academic twitter was very helpful for that, to find out what they were doing and profile people and amplify people who are doing amazing things. What works best is when something specific, if you say how should we change this class, you get silence chicken. If you say here are the 10 learning outcomes that our department requires us to write, do you have anything else you’d like to add? Maybe work with a partner and come up with your own additions to these required learning outcomes? Students come up with beautiful, soaring, inspirational things that just make you aware that if they’re allowed not to be cynical, students want to have agency and want to have something that will help them in the rest of their life, it’s pretty scary to think about the world out there. And they want and need these things and want to be participants in the shaping of their own life and in their own agency.

Christina: And you’re reminding me too, that there’s this widespread movement right now to rename office hours, to call them student hours, because students are so used to going to the office being a punitive experience. And student hours really welcomes them to come with maybe more than just “I’m having difficulty with this problem set.” “But I want to talk about my career, what can I do with this degree? Or I’m having difficulty with x? Can you help me with y?” I think it really better serves them as well to rename it student hours to show that we actually really want you to come and we really want to talk to you. I don’t want to just sit alone in my office. [LAUGHTER] I want you to be here.

John: What are some other activities or ways in which we could give students a bit more agency? You mentioned the reflection on the syllabus and rewriting a new syllabus at the end. And you also hinted at having students be engaged in the syllabus itself. What are some other ways we can do this during the course of the class?

Cathy: I often quote our friend Jonathan Sterne, who teaches at McGill University because this is the most counterintuitive one. Jonathan teaches various versions of mediasStudies, media information, technology, and disability studies. He himself had throat cancer, and he talks through a voice box that he himself helped to design… a remarkable human being. He teaches classes of 400 to 600 students, and people say there’s no way… how in the world could that be active learning? He hands out index cards and at the end of every class students write down an answer to something he asks. He might ask “What did we talk about in class today that you’re still going to be thinking about before you go to sleep at night? And if there was nothing, what should we have been talking about?” Today’s media if you don’t have anything to say about media in the modern world, something’s misfiring. He has a blog about his learning, and he’s charted how well students have done since you’ve done these simple exercises of having students report back after every class. He also works with TAs who have special sections, and they take the 15 cards from their section, so they know what questions the students have before they go into the session. And then he uses some of those to spur his next assignment. He does this kind of cool thing where he spreads it all the cards and says, “Well, John said so and so. And Deborah said so and so. And Rebecca said, so and so. And Christina said so and so.” So he makes it interactive. He also does an incredible thing. He says, “With this many students, I have to use multiple choice testing, and I know how impoverished multiple choice testing is.” So he sets his students a creative assignment each time, sometimes it’ll be a piece of notebook paper, sometimes it’ll be an index card, and he’ll say, “You can write any crib sheet you want, go back, and you can do anything you want from the semester to help you do well on the multiple choice exam.” And what he knows is what they write on that crib sheet is the learning, right? It’s not filling in the ABCDEs, it’s the learning he also leaves some portion, it might be 5%, might be five points, it might be10 points, depending on the system he’s using. He’s Canadian university, so it’s a slightly different system than in the US. But he leaves some amount free, and students hand in both their multiple choice exam and their crib sheet. And he gives extra points for the crib sheets. He’s even done art installations with some of the crib sheets that students have done. But the point is the way you review a class and organize the knowledge onto some very prescriptive sheet, and he says the prescription is extremely important… he’s a composer as well. So he knows how it’s important is to have rules and to play with those rules. And it’s almost like a game. That’s where the learning is happening. So even in the most restrictive situation, you can still do active learning and learning that’s meaningful to how your students learn and how they retain and how they can apply that learning later.

Christina: A colleague of mine, Siqi Tu, also has her sociology students come up with some of the questions that will be included on the final exam. And you can do this with a multiple choice or a long answer type of exam, and the students develop a question. And then when they submit their questions to the professor to review, they also need to include what skills are we assessing with this question? And is this the right answer? What are the various ways you could get to the right answer? And if the right answer is D, then why are you offering A, B, and C as other but wrong answers? Why are they not the right answer? …things like that, to get students to have this kind of command over what they’re learning, why it’s important, what is worthy of being assessed, and how to go about assessing it and testing that knowledge. They have a lot more agency then in how they are being evaluated. And so their expertise is also being solicited there. And the majority of the learning is happening in creating the question and explaining all of the pieces of how that question has been crafted and what the right answer is and why. And then, at the end, she includes at least a portion of the student-generated questions on the final exam. And so it not only gives them great exam prep, but then they also know better what to expect. And they have more agency and control over how they’re being evaluated.

Rebecca: I really enjoyed some of the titles of the chapters in your book, for example, “group work without the groans.” [LAUGHTER] And I thought, yes, group work without the growns. So they resonate.

Cathy: I used to consult quite a lot with business. And just because of that I helped create the Duke Corporate Education Program, which was for returning executives, when I taught at my previous institution. And the number one expense of management experts who are brought into corporations to help manage more effectively is to help them with group work. And in classrooms, sometimes even in sixth grade, we say “Work in your groups and then do something amazing.” Well, no, we know what’s going to happen. That one person who raises their hand, they were one of the three who raises their hand, and in the group, they’re the one that does all the work, then there’s somebody else who kind of goofs off, and then there’s somebody else who does nothing at all. And we know those patterns, and it’s horrible for all three, it’s horrible for the person who always steps up, they’re not being pushed, and it’s horrible for the person who does nothing or the person who goofs off. So when I do group work, I have students write job descriptions. They write out job descriptions for who is going to do what in the group and I also love to have them do an exercise that I call superpowers. What’s your hidden superpower? What’s your three things you do that you think have no relevance at all to this group but that you know you do really well? It might be playing video games. I’ve had people say they were a clown. I found out that my executive director of the program I run was a professional clown. I didn’t know that. She’s a gorgeous young woman and she was a clown. She’s the last person I would have thought of who would be a professional clown. People have these skills, and what they find out in a group is those skills are not irrelevant, because you’re talking in group work not only about coming up with a product, but about interrelationships. And how you can make all the different parts of your personalities work together coherently to create a final product. Also, I have students put that on their resume. I have them look online at what employers most prize. And it turns out collaboration… duh…because that’s what they spend their money improving. Somebody who can be a great collaborator is somebody you want to hire in a job. So my students can not only put that on their resume, but have a wonderful example. You don’t have to say it’s in a classroom. I worked on a project with four others, and we took that project in from idea to implementation, and my role was the firestarter. That’s a term from computer scientists. I’m especially good at coming up with new ideas and presenting those ideas to a group. Fantastic. And then they don’t grown, they realize they’re learning a skill, not being put back into a pattern that they themselves hate and are embarrassed by or resentful of.

Christina: And I think in addition to telling them why group work is important, that they’re going to end up working in groups for the rest of their lives. Everything I’ve done in any job has been collaborative to some degree, and mostly a lot, like really collaborative. And so I just kind of tell them, “This is not busy work, this is actually good practice for the rest of your life, because at the end of the semester, you’re not going to work with these people anymore. But if you’re in a job, the only way to leave that group is to leave that job and the stakes are so much higher. So first of all, this is good practice.” But I think also, we sometimes neglect to offer students the structures that they need to feel confident in their grade for group work, and to feel confident going into group work. So from the get go, giving students structure for the group work, like a checklist of jobs or asking them to come up with a checklist of tasks that they need to complete, assigning roles, like putting a name next to each item on that checklist, so that it’s clear who is doing what. Teaching them a skill like that is teaching them how to delegate authority, how to be a good entrepreneur, a leader, and pointing that out to them that by creating this checklist and putting everyone’s name next to everything, you’re delegating authority, you’re learning these leadership skills that you need in the workforce, and helping them to understand that it is absolutely okay and totally normal to feel social anxiety before going into a group, particularly after a pandemic, when we really lost the ability to make small talk.

Cathy: It’s exhausting. [LAUGHTER]

Christina: It’s exhausting. It’s exhausting. It is we lost that skill. Oh my goodness,

Cathy: Today happens to be a Monday and I had brunches on Saturday and Sunday and took a nap after each one. I can’t remember how to do brunch anymore. [LAUGHTER] And we’re all in that situation.

Christina: It’s true. I just wanted to add one more thing, which is that it’s not just social anxiety, but it’s also anxiety about grades and grading. And we have a whole chapter about grades. Because no one likes grading. No one likes grading. And I think it’s important too for students to know that they’re being graded and assessed fairly on group work. I’m so having that checklist of roles and turning that in at the end to show who did what is really important. And inevitably, it’s funny because we also put faculty into groups …that transformative learning the humanities… where I work, and this inevitably happens with faculty too, that they get really anxious about working when the ideas aren’t gelling together. And I think it’s really important for students to know how they’re being graded. And if someone’s ghosting or someone’s not pulling their weight or not showing up, give everyone else extra credit for helping to make that work, helping to reach out to that person who’s not showing up or ghosting: “Are you okay?” And a lot of times that person is not okay. And they needed someone to reach out and ask if they’re okay, they need that support. So offering the students who unclog a problem, you can give them a plunger award, literally, that happens in a lot of groups, or do something to recognize someone who goes above and beyond to help resolve those kinds of conflicts and issues, rather than it feeling like “Oh, but my grade is being hurt by someone else.” I think it really helps to foster collaborative community and a learning community where everyone is important. Everyone is valued and everyone needs to be okay for the group work to be successful. How can we help our colleagues, our peers.

Cathy: Around 2005 to 2010 years for my organization HASTAC which I co-founded in 2002… NSF now called it the world’s first and oldest academic social network… we created a wiki and we went to this guy in a garage and asked him if he would help us create a wiki and then the next year he launched Wikipedia… that was Jimmy Wales. [LAUGHTER] So that’s how old, that’s how ancient this is. But with that world, we were dealing a lot with open-source computer programmers. When they would do a job, when a job would be posted on Stack Exchange or another open source site, they would have to find a partner that they’d never worked with before and know if that person was reliable, somebody who could complement their skills, and so they do a badging system. And I worked with the Mozilla Foundation on creating badging systems, where you would write down all the criteria that you need to accomplish a job and you never gave a negative, you never give a negative, you just give somebody a badge if they did something great in doing that, and then somebody else who comes by and wants to look to see if they want to work with that person sees where they’ve been given the right badges and says, “ooh, yeah, those are the things I do poorly and those are things what you do well.” Again, HASTAC calls that collaboration by difference. Not everybody has to do everything perfectly, but you have to know, you have to have an inventory of what people contribute. So in my classes, also, when I’m doing group work, I’ll have students not only write their own job descriptions, but write a list of the qualities they think are most important for the success of the group as a whole. And after every week, when they come together, I’ll have them give badges to the people. I don’t even say how many, what percentage, just give a badge to someone who you think really showed up this week, and give a badge in a different category. You don’t have to tell the person who never gets a badge from any of their peers that they’re not pulling their weight. And then that can be a first step, as Christina said, to doing something like reaching out and saying, “You must feel terrible that nobody’s given you a badge in anything. What’s going on? Is it indifference? Is something going on your life? What’s happening?” Yeah, and it doesn’t have to be punitive. But when you see that none of your peers are rewarding you, sometimes it’s like, “Darn them. I’m doing all the work. But I’m a shy person. So I just make the corrections.” That often happens among computer programmers who often are not the most voluble, personable, people. They used to wording, we’re doing code? And someone will say, :”No, no, no, I was fixing all the code you didn’t see, you just didn’t look and see that I was fixing all the code.” And then you can make an adjustment to and it’s an incredibly important adjustment. I’m working with somebody now who was Phi Beta Kappa, three majors, straight As, et cetera, et cetera, and never spok in college. And when she hears these methods, she says, “This would have changed my whole way of being in college.” Instead of feeling shame of all the people who should not feel shame, she felt ashamed that she wasn’t contributing. And because she was never offered an opportunity to until she wrote this brilliant final exam or brilliant final research paper when her teachers knew but in class, she felt like she was failing. That’s horrible. That somebody that brilliant would ever feel like they weren’t doing a great job.

Christina: I think it also asked how to get students excited. And thinking about what Cathy just said, I like to frame group work as an opportunity to practice being a step-up or a step-back person. So if you’re normally a step-up person, like everyone loves that you’re the first to volunteer to help. It’s great to be a step-up person, but sometimes that doesn’t leave room for stepping back and taking into account all of the things that have been said, and reflecting on the larger picture and finding the forest through the trees. And so I invite students to try to practice if you’re generally a step-up person, try being a step-back person. And if you’re generally a step-back person, try being a step-up person and see how it goes. And I also put all of the really loud step-up people in a group together, and I put all of the really quiet step-back people in a group together, because at some point, they’re going to have to talk and the step-up people need a way to regulate who is talking. And so I think a lot of us try to distribute those people through groups, and that can really change group dynamics. So I like having them all together in various ways to feel comfortable being among peers and navigating those roles and being more aware, calling to mind those roles before group work, so they could get excited about trying something new, and recognize that this is always just practice.

John: Will these methods help to create a more inclusive classroom environment?

Christina: These methods are inclusive because they solicit participation from every single person in the room. Inventory methods achieve total participation, that’s a term from the American Psychological Association, 100% participation, not just the hand-raising few. So these methods are inclusive.

John: There has been a lot of criticism recently concerning the way in which traditional grading systems cause students to focus on trying to achieve the highest grades rather than on learning. Do you have any suggestions on how we can focus student effort on learning rather than achieving higher grades?

Christina: Shifting the focus from grades to learning? A very quick answer would be thinking along the lines of Carol Dweck and using a growth mindset… that we are all learning… that you can do more assignments that are completion-based or labor-based to a satisfactory degree rather than A, B, C, D, F. And I think that one really great model is from Debbie Gail Mitchell, who teaches chemistry in Denver. She decided that an 80 is achieving proficiency. And the goal is to achieve proficiency. And so if you get an 80 on an exam or an assignment, then you receive the total number of points for that exam or assignment. And there’s a total number of points to achieve for the whole class. And so assignment or exam adds up to that. And so students stop grade grubbing or worrying if they’re getting an 83 or an 84. And they focus more on learning.

John: We always end with the question: “What’s next?”

Cathy: At one of those brunches This weekend, I had the privilege to meet someone named Matt Salesses, S-A-L-E-S-S-E-S, who is a novelist who just took a job in the MFA program at Columbia, who’s also written a book called Craft. And the book looks at how we teach writing in writing workshops by looking at novels and noticing how often white-authored novels don’t tell you the race of the white characters, but do tell you the race of the non-white characters, and how much craft through all fields as well as in writing workshops often assumes a putative expert in a putative subject as being white. And then everybody else gets defined. So even when we do the terms like diversity and inclusion, there’s an implicit grounding that the person who has craft and earns that craft is going to be from the dominant race. And it’s a really interesting book that looks at the Iowa Writers Workshop and the principles by which it was written up. You could have done craft as a way that higher education was set up. So many of the people who started and set up the metrics for higher education in the late 19th century were in fact, eugenicists, they really believed there was a biological reason for racial superiority and created a system that reified that prejudice that they had. So that’s just a parting. I just happened to be at a brunch at one of those exotic face-to-face human real experiences this weekend where I met an astonishing person with an astonishing book that I’m thinking about more and more. So that’s a what next for me as of yesterday, but that’s the wonderful thing about active learning is yesterday always has something interesting you can learn from.

Christina: My what next is I’m working on an article right now with a colleague, Josefine Ziebell. We’re both sound studies scholars, and also interested in pedagogy. And we’re looking at the school-to-prison pipeline, and the ways in which school soundscape mirrors the carceral soundscape, and how to give students more sonic agency. And so thinking about silence in the room, thinking about voice, and in what ways speaking up can challenge authority and how that can cost you your life in the worst scenarios. And also just like bells ringing, the time ticking ways in which school soundscape can contribute to the school-to-prison pipeline by regimenting everything about a person’s body and depriving them of their sonic agency, where making noise is considered inappropriate when that is exactly what we need to do to transform these institutions. And so I guess that just studying that I find really interesting right now with Josefine.

Rebecca: Well, thank you so much for such fascinating conversation today and things to think about and wonderful teasers for your book.

Cathy: Thank you very much.

Christina: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

248. Reframing Academic Expertise

Professors are generally represented in popular culture as white male experts who dispense knowledge to their students through lectures. Young female professors are often encouraged to portray themselves as authoritative figures, even when this role does not reflect their personalities and their educational philosophies. In this episode, Rebecca Scott joins us to discuss how she has rejected this stereotype by sharing vulnerability and building classes that rely on the co-creation of knowledge.

Show Notes

Transcript

John: Professors are generally represented in popular culture as white male experts who dispense knowledge to their students through lectures. Young female professors are often encouraged to portray themselves as authoritative figures, even when this role does not reflect their personalities and their educational philosophies. In this episode, we discuss how one professor has rejected this stereotype by sharing vulnerability and building classes that rely on the co-creation of knowledge.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Rebecca Scott. Rebecca is an Assistant Professor of Philosophy at Harper College, and also a guitarist and vocalist in the band Panda Riot, which just released their fourth album. She’s also the author of a chapter in Picture a Professor, edited by our friend Jessamyn Neuhaus from SUNY Plattsburgh. Welcome, Rebecca.

Rebecca S.: Thank you. I’m excited to be here.

Rebecca: Today’s teas are… Rebecca, are you drinking tea?

Rebecca S.: I’m drinking coffee.

Rebecca: Oh, a coffee drinker….

John: Well, we drink coffee too, once in a while.

Rebecca: Very occasionally, but not on this podcast. [LAUGHTER]

John: I am drinking Irish Breakfast tea this morning.

Rebecca: Oh, it sounds like a good theme. John. I have English breakfast this morning.

John: We’ve invited you here to discuss your chapter on “Reframing Academic Expertise through Vulnerability and Metacognition” in Picture a Professor and some of the other work that you’re doing. But before we discuss this, could you talk a little bit about your institution, and the courses that you teach?

Rebecca S.: Harper College is a two-year college, or community college, in the suburbs of Chicago. And, because we’re a community college all my classes really are introductory classes or don’t have any prerequisites anyway. So I mostly teach Intro to Philosophy, Critical Thinking, and Ethics, then I occasionally get to teach Biomedical Ethics and the occasional honors class. And in the fall, we have a new program that’s a social justice studies distinction. So I’m going to be teaching an Intro to Philosophy class that’s specifically for that Social Justice Studies program at our college.

Rebecca: Your chapter title is really intriguing in Picture a Professor. Can you talk a little bit about or give us a little teaser on how you reframe academic expertise through vulnerability and metacognition?

Rebecca S.: In that chapter, when I was approaching this question of how do we address these kinds of biases against professors who come from marginalized identities, the way that I came to this question is, when I was first starting out, people would give me a lot of advice about, “Oh, you know, you’re young, and you look young, and you’re a woman, and you have to be really careful.” And I feel like a lot of the advice I got was that I needed to be like, strict and I needed to be like, hard, and I needed to take on this authoritative stance so that people would give me credibility. And it just felt really not who I was. It just wasn’t me. And it works for some people, I think, and that’s great. I’m not in any way saying that people shouldn’t necessarily do those things. But for me, it really just was not the way that I wanted to teach. And so I’m always trying to figure out how can I have authority and credibility in a way that feels authentic to who I am. And so this got me thinking about the ways that part of the problem with the sort of stereotypical image of the professor is not necessarily that we don’t have enough different kinds of people occupying the role of the professor, but that the whole concept of the professor is part of the problem. And so in my chapter, I’m thinking about how can we think about not just having different kinds of bodies and people occupying this social role, but what ways do we need to actually change the social role in the first place? And so I think that the kind of epistemic authority that a professor has is often this individualistic, like knowledge is a kind of property that is sort of won through this genius and hard work or whatever, and not thinking about the ways in which knowledge is constructed and maintained in communities. In my chapter, I’m thinking about how do we teach in a way that presents academic or professorial or epistemic expertise in a way that acknowledges the ways in which knowledge comes about in and through communities. And so the vulnerability and metacognition are sort of like two strategies. So for me, like kind of leaning into the vulnerability and modeling epistemic humility when you don’t know things and being engaged in the process of coming to know rather than seeing knowledge is something that you arrive at, like “Now I’m a professor, so I know all the thing and you are the students and you don’t know the things and let me figure out a way for you to have the knowledge that I have,” …because even in, I think, constructivist or collaborative models of education, I think there still tends to be this like, individualized aspect. And there isn’t always like a true sense in which the space of the classroom is co-created, co-constructed by the community that we are all a part of… the professor and the students. That’s the sort of overall approach that I took to the challenge of the question.

John: How do you sell this to students? Because that’s an approach that they may not be familiar with based on their past educational experience.

Rebecca S.: Yeah, this is always, I think, a really big challenge, especially when you’re trying to do something that’s sort of radically different, and you want to do it in 16 weeks, and they’ve had years and years of education going against it. I also do things with games and play and game-based learning. And I actually have found that cultivating an attitude of playfulness can go a really long way towards breaking down some of the ways of being that students have been sort of trained to be in in the classroom. And so I think that there’s a way in which, if I can open students up to laughing and having fun, and just getting in a different kind of physical and mental space, then can sort of start to chip away at some of that. And I don’t think it’s possible to do in one semester. This is another thing about the problems with stereotypes. There’s not some magic pedagogy that’s going to eliminate racism, you have to accept the limits of what we can do in one class as one instructor for one semester. [LAUGHTER]

Rebecca: I really like some of the work that you’ve been doing related to games in the classroom. And we’re hoping that you might expand upon that a bit. And one of the things that struck me about… I think it was a blog post that you wrote about Dungeons and Dragons in the classroom… that you talked about how the people participating in the game, create the world and create the experience together, and that the knowledge of the space is created together. And it seems really tied to the work that you have posed in Picture a Professor. Can you tell us a little bit about how you’re using Dungeons and Dragons to teach ethics?

Rebecca S.: For me, when I first played Dungeons and Dragons, I was in my 30s already. I’m not one of these like lifers forever with DnD, but I realized how many parallels there were with the players and the Dungeon Master and the teachers and the student, because the Dungeon Master has a particular role to play, or the Game Master in a role-playing game. And it’s an importantly different role than the role that the players have. And I think the same is true, it’s not that the teacher and the student are the same in the classroom. But what’s really exciting about role-playing games, or at least tabletop role-playing games like Dungeons and Dragons, is that what the game is about, like what the values are… Is this going to be a game where we mostly focus on strategy and try to get as much money as possible to buy all the cool weapons or whatever? …like, that’s one way the game could go? Or is this like going to be a funny game where we’re just like joking around and getting into high jinks and whatever, are we going to explore some really serious things with identity and characters, like all of those things can be a Dungeons and Dragons game or a role-playing game. And it’s not something that can be dictated from the Game Master. It’s something that comes together through the creative, collaborative storytelling of the game itself. And so I think that there’s just so many parallels with teaching, where the teacher does know more things, and does have a responsibility to make sure that everyone’s involved, everyone’s included, everyone’s enjoying, everyone’s achieving the goals. There’s a certain responsibility of the teacher or the Game Master. And there’s a certain kind of knowledge and a certain kind of expertise. But what the game is, is fundamentally co-created,

John: Do you use that directly or indirectly to help share your teaching philosophy with students?

Rebecca S.: I do always try to share with students why we’re doing what we’re doing. And I’m trying to be intentional about making transparent, especially when we’re doing something weird.you knowe… I’m like, okay, alright, come on, like, you guys, humor me, we’re gonna try this thing. It may totally fail, but we’re gonna try it. So I’ll often talk to students about my teaching philosophy. I’m not sure that I’ve ever actually made that particular idea explicit, and maybe I should. So I think it’s more implicit. But now you’ve made me think that I probably ought to make it explicit.

John: Or maybe have them think about it and make the connections themselves.

Rebecca: Can you talk about how Dungeons and Dragons has unfolded in your ethics class, and what that assignment or activity actually look like?

Rebecca S.: So the way that I’ve constructed it now is that it’s the last four weeks or so of the semester. So it’s after we’ve covered a bunch of material. My dream is to figure out a way to make it the whole semester, but I haven’t quite gotten there yet. So we have several units and we study different philosophers, different ethical theories. So we do Aristotle and we do Mozi, an ancient Chinese philosopher, we do have Simone de Beauvoir and existentialists, they see a bunch of different ethical theories. And then for the last four weeks of the class, in groups, they create a character based on one of the philosophers that we’ve studied or one of the theoretical frameworks that we’ve studied. And so they have to come up with a backstory, like what kinds of experiences would someone have that would lead them to have an existentialist ethics or like, what kind of a person would be a Kantian and they have to pick a class. So they’re like, would Beauvoir be a wizard or a Paladin or whatever. And so they have to pick a class. And then they create their character. And in Dungeons and Dragons, there’s an alignment system. I have my own kind of alignment system, so they have to say whether they’re more focused on contextual factors or universal principles, that they have to pick their alignment for their character. And then we play the game. So I describe a scenario and then they have to say what choices their characters would make in a given situation. So I’m like, okay, you’ve all been called here by the Queen whose son has been kidnapped, and she sends you off to go rescue her son. So they’re all like, “Yes.” But then as they’re leaving the towns, and people say, “Actually, we don’t like the queen, we don’t want a monarchy.” And then they have to decide like, “Okay, do we save the queen’s son, which we made a promise to do, but that’s going to perpetuate the monarchy or do we help these rebels who want to bring in democracy.” And so there’s different decision points along the way, and they have to decide what their characters would do. And then they write reflections at the end of each day of the game, where they say, what decisions they thought fit well with the ethical theories and which ones they think could have been better and why. So they do some post-game reflection.

Rebecca: How have students responded to this kind of an experience?

Rebecca S.: So I’ve done some surveys and things just to ask students, and I don’t think I’ve really gotten anything really negative, everyone seems to think it’s fun. They say that it’s really helpful as a review and as application of the ideas. Some of the students get super into it. So that’s always fun. And I think that’s true with any activity you do. Like, you’re always going to have those students that are just like, “This is my thing” and run with it. So that’s really fun. And those students are not always the students that are necessarily the most engaged in the other parts of the class. And I think that’s really a benefit too of doing these kinds of different sorts of activities, because you have a student that maybe hasn’t had positive experiences with academics so far, but they hear Dungeons and Dragons and they’re just ready to go. And then even the students that are not as familiar with it, they have fun with it, too. And some of them kind of get into it unexpectedly. I also don’t force anyone to play. So one representative from each group plays each day, but I always make sure everybody has the chance to play, but they have the option if they just want to observe and do the reflections, that’s fine. So I think that helps with some of the potential discomfort that some students might have. But what I find overall is that the humor of it is really interesting, and really solidifying of both knowledge and community. So for example… I think I talked about this in the blog post, because this is just my favorite example… but I had a student who was playing a character inspired by Kant, and one of Kant’s principles is you’re never allowed to lie. And so they were sneaking into this goblin cave and the Kant characters, like, “I can’t sneak, it’s a lie, it’s deceptive.” So he goes in and announces that they have arrived at the goblin cave, and everybody laughs because everybody’s in on the joke, because everyone knows that Kant says, “You’re not allowed to lie.” And so there’s this kind of inside humor that is possible. That really is like, “Oh, we learned something this semester, we all now get this Kant joke that, at the beginning of the semester, no one would have understood.” It creates this sort of in-group thing, but not in a negative way. But like a positive way, like we’ve all learned this together. And we have this shared humor now. That I think is really fun.

John: In one of your tweets, you mentioned that you were planning a course that would involve some world building, could you tell us a little bit about that?

Rebecca S.: So I agreed to do some pedagogy through world building for this book that’s going to be coming out. It’s for a case study for the book. And it may or may not actually even be included in the book, but a bunch of faculty are doing some world building and writing about it, and then we’ll see what happens with the book project. So I agreed to do that. And then I’m also doing this class for the Social Justice Studies program. So I was like, “How can I match these up?” And so I was thinking about how to teach with world building, and the first thing that came to my mind is Plato’s Republic. So Plato’s Republic is a dialogue by Plato where he is exploring the concept of justice and he imagines this ideal city that ends up not being ideal in the end, but he’s trying to envision what would justice look like in this city and so he creates this world with its own myths, and with its own laws and rules and education system and marital practices. So I was thinking that it would be fun if we started with Plato, and then had students create their own just world where we think about what is justice? And what would justice look like in all these ways. This is the plan so far… there’s a site called World Anvil that is a world building site that people use for making role playing games, but also novels and things like that. And it’s kind of like a wiki. So essentially, the students are going to be creating a wiki of their world. And so we’ll have to decide what we want to focus on, because obviously, we can’t do everything. So we’ll have to think like, do they want to talk about education systems, do they want to talk about criminal justice systems, do they want to talk about religion? So they could come up with their pantheon of deities, if they want. What would religion look like in a just society? Would everyone have the same religion? Would there be no religion? The topics will be student led, I’m going to have a list that they can pick from, then the readings will be determined by like, we decided to do religion, and I’ll give them some readings, and then the assignment will be to build that part of the wiki, the religion part. And recently, I just started thinking, we’re not going to get very far in one semester. So what if this was like a project that the next time I teach this class, they take up, and we keep building the world. So, it may end up being a long project where each class picks up where the last class left off, which I think would be really cool. This is still in the planning stage,… the next couple of months to really nail it down. But that’s where I’m at so far. [LAUGHTER]

Rebecca: I really love the idea because it involves students in a lot of decision making, and really contemplating the ideas about social justice. So they would need to have some background knowledge, and then have discussions and co-create and co-decide on things and so it seems like a much more active way of engaging with the material, then maybe a traditional paper or other kinds of activities, like quizzes and stuff might have.

Rebecca S.: Yeah, and I think having this project, it’s kind of problem-based learning, in a sense, I guess. It’s a little bit different. But having that sort of shared project, I think, also hopefully will create a different kind of engagement and motivation.

John: It sounds fascinating. Can we go back, though, to your chapter in Picture a Professor? You’ve talked a little bit about vulnerability and how you share that. But could you talk a little bit about how you build metacognition into your classes.

Rebecca S.: So philosophy classes in general focus a lot on class discussions. And I think sometimes students think, “Well, what do I really learn from a discussion?” So going back to this idea of the professor is the one that has the knowledge, like, “Well, I’m here to learn from the professor and not from my classmates…” or they think that a discussion is just saying things, just whatever comes up, and I promise this is going to get to metacognition. And so what I focused on a lot is thinking about how to improve class discussions and how to help students think about what they’re learning from class discussions, by thinking about conversations in terms of academic moves. I didn’t invent this idea, but I’ve really tried to take it and develop it. So this idea that when you contribute to a discussion, you’re not just saying something, you’re asking a clarification question, or you’re posing a hypothetical, or you’re disagreeing with someone, or you’re agreeing with someone and saying why, or you’re connecting to your own experience, right? So there’s these specific things that you’re doing. You’re not just saying things, you’re doing things in a conversation. And from a social justice perspective, you can also do things like welcome someone into the conversation with a question or exclude someone or silence someone. So the idea that saying things does things is, I think, one of the most important ideas I want students to come away with. So I do a lot of work with having students identify what moves they’re making in a conversation and thinking about and reflecting on what kinds of moves are most productive, or are there any moves that we don’t want to make, like fallacy or ad hominem, like illegal move. But having students reflect specifically on what sort of contributions they’re making or how they’re moving a conversation forward. And this is not just for conversations, but also for writing, or you can also identify moves that people are making when you’re reading a text. And this is also a way of recognizing the communal nature of knowledge because one of the move can be to thank someone for helping you see something in a new way. Like that’s a move too, like showing gratitude to someone else, or acknowledging someone’s contribution or summarizing what someone else said or asking them to clarify. So there’s a lot of different activities that might be involved, but they’re all about sort of metacognitively reflecting on academic discourse, whether that’s written or spoken, and specifically identifying the ways in which we see through that, that this is us together creating a community rather than each individual person gaining knowledge on their own.

John: So how do you implement that specifically in terms of students reflecting on that? Do you have them engage in a conversation and then reflect back on their participation? Or are there other techniques that you use?

Rebecca S.: Yeah, I’ve tried a bunch of different things. So I’ve tried a game version, where it have the different moves on cards. And it’s an idea I got from Ann Cahill at Elon University. So she actually had a deck of these cards. And then I’ve tried that where I deal out the cards, and then they have to look for an opportunity to play their specific move. And that works kind of okay, but I haven’t had complete success with that, because students often find it difficult to find the moment for their card, or they want to say something and they don’t have the card for it or whatever. So I think that’s fun as a way of introducing it and practicing it, but it can interrupt the flow of the conversation some. What I did recently in an online class this past semester, is I actually divided the moves into different levels. So things like “connect your own experience,” most students can do that relatively right away. Whereas “identifying unstated assumption” like that’s a really hard move that takes a lot of work and practice. So for their first paper, I use Perusall, and so they comment on the text and respond to each other. But in their responses to one another, they have to identify the move that they’re making. So they’re responding to a classmate and they’re saying, “Okay, I’m going to agree and give a new reason. I’m going to disagree and explain why or I’m going to propose a hypothetical or whatever.” So I have them actually, in their discussion posts, identify the move that they’re making before they make the move. So that’s one way I’ve done it. And then I’ve also just done some things where I have them just write reflections on the discussion and identify moves that have been made or moves that haven’t been made, and why haven’t those moves been made? So things like that, also.

John: Do you have them use tags in Perusall for the types of moves that they make?

Rebecca S.: That’s a good idea, I have not, I only a very basic Perusall user. I was using Discord for awhile. And this is really fun, I actually had to make custom emojis for the different moves. So that was a fun activity at the beginning where they had to come up with an image to associate with the moves, it didn’t really play out fully in terms of the way that I envisioned it. But I think there’s still promise with that approach as well, I just need to pursue it more.

Rebecca: I like the idea of actively having to be conscious of what kind of move you’re making while you’re making it. That does seem like it may work a little bit better in an asynchronous environment where people have time to think about what move [LAUGHTER] they’re making, rather than in a synchronous context. When you were talking about conversations, it was reminding me of a really interesting conversation that we had, on our campus, with our workgroup on accessibility practices, with some students with disabilities, who identified that conversation, like classes that focused on discussion, felt really inaccessible to them or were hard to follow because they were having a hard time pulling out what to take away from the conversation. So you led this little segment about that. So it made me start thinking about how could we slow things down a little bit to be a little more cognizant of what we’re doing and maybe give time to digest what’s happening rather than the rapid fire and not being able to keep up. Although it maybe isn’t a natural flow of conversation, it does make you think more about what it is that you’re doing before you’re acting.

Rebecca S.: Yeah, that’s a really great point. Now that you mention it, I do think that it is often more successful in asynchronous classes. And in the synchronous classes, it’s often more of a after-the-fact reflection, but I think that it could be useful to have students plan out their moves in advance of the class. It’s like in preparation for the synchronous discussion and then, see then maybe not just like, here’s your card, make your move. And everyone’s like, “I don’t know, you have to pose a hypothetical right now?” But if everyone knew the moves that they were supposed to make in the class that day in advance, I think that would actually work really well. So I’m going to steal that and do that. [LAUGHTER]

Rebecca: Yeah, that might be interesting. I think one really good accessibility practice for any kind of presentation or activity is for people to know what’s going to be expected of them in advance so that they do have time to think so I’d be interested to hear, whether or not if you do that, how that plays out. So I know inclusivity is important to you. Can you talk a little bit about how some of the practices that we’ve been discussing today promote inclusivity? Or the ways that you think about setting up activities to make sure that people feel included?

Rebecca S.: Yeah, so I’ve been thinking a lot about inclusivity and creativity and vulnerability and playfulness. And I think it’s always tricky, because I think that sometimes we have this idea that there’s some sort of ideally accessible and inclusive class, that if we just keep opening and opening and opening and opening that somehow eventually it will include everyone equally. And I don’t think that that’s possible, because I think that when you really get into the concrete details of things, something that works really well for one student might be more difficult for another student, and how do you balance and weigh these kinds of complicated decisions? Nothing is ever just straightforwardly more accessible or more inclusive, I don’t think. So, I think it’s much more complicated than that. So the way I’ve been thinking about it is actually through a philosopher that I mentioned in a Chapter, Jose Medina, his book is called Epistemology of Resistance. And he talks about the need for epistemic friction. He talks a lot about the ways in which people who have a lot of privilege don’t encounter enough epistemic friction. So there’s this way in which things are too smooth and too easy, and you’re not challenged enough. There’s not enough resistance that you face if you have a lot of privilege. And he talks about the kinds of… I don’t want to say benefits, because that’s not quite right… but, the ways in which having a marginalized or oppressed identity can create the opportunity for developing certain virtues, certain epistemic virtues, not to say that it’s a good thing, like, obviously, it’s a bad thing. But what he does is he kind of flips things on its head where he says, we often think about privilege as benefiting people who have privilege. But there’s also ways in which privilege isolate you and prevent you from being able to know things or learn things or develop certain skills. So to get back to inclusivity, I’ve been thinking about being a teacher is about managing epistemic friction. So the idea that at certain times, certain students in certain contexts actually need a little more friction, and certain students need a little less friction. And so rather than thinking about creating some ideally open space, it’s about managing the kinds of friction that students run into. So some kinds of friction are unjust and should not exist. They’re not even friction, they’re just obstacles. But some kinds of things that are difficult or uncomfortable or challenging can be really good and beneficial for learning. But it’s really tricky to know which students need what and you always have to be very careful about presuming that you know what students need, of course, but I’ve been thinking about designing classes in terms of eliminating unjust epistemic friction, but creating opportunities for certain other kinds of friction, and thinking about accessibility in terms of that, like, is this a productive kind of challenge? Or is this an unfair or unjust kind of challenge? Because it’s not about whether it’s difficult or not. Learning is difficult. It’s like, is it the right kind of difficulty? Is it the right kind of challenge? Is it fair and just and promoting of the learning rather than the opposite of that, if that makes sense.

Rebecca: I like the word choice that used of obstacle versus a challenge. An obstacle is something that it shouldn’t be there… [LAUGHTER] versus a challenge is something that we would hope students actually do experience as students, because challenges can help us learn.

John: How do you set students up for this, because some of those discussions could be somewhat challenging for people who don’t want to have their beliefs challenged, who have really deep beliefs, and are resistant to learning new things, [LAUGHTER] or new experiences.

Rebecca S.: This is where I think that modeling some epistemic vulnerability is really important and humility. And so, for instance, I teach critical thinking, and I had this activity all designed, and I was excited about it and I thought it was gonna be great. And it was terrible. And I said to the students, at the end, I was like, “Well, that didn’t work at all. Like that was a disaster.” And then I’m like, “okay,” then it was like, “You guys, I have another class in 15 minutes. I have another section.” I’m like, “Quick, what do I do? How do I make this better? How do I save the next class from this terrible disaster?” It wasn’t that terrible, but it was just awkward and didn’t really work. And then they were like, “Oh, well, the problem is that you thought that we wanted to talk about these things. But we actually don’t care about these issues that you think we care about. We want to talk about these other things like ‘Do aliens exist?’ or whatever.” And I was like, “Oh, okay.” So in the next class, they ended up debating whether a hot dog was a sandwich. And I thought, this is so silly. Why are we talking about this? But they got really into it. And I told the next class, by the way, the last class sucked, and I changed everything. So here we go on the fly. And so I let them know that I made a mistake or I was wrong. I miscalculated how things would go and I made a change. And now we’re going to try it. And at the end of that class, there was this student and he had changed his mind about whether a hot dog was a sandwich. And he was really resisting admitting that he changed his mind. And so it was just like, I’ll call him John. John. His name’s not John. So I’m like, “John, you were wrong about whether a hot dog was a sandwich. He was like, “No.” John, you were wrong, right? And then everyone’s like, he wouldn’t say it, and I’m like, “Can you say ‘I was wrong?’” …but it’s really funny at this point, everybody’s laughing. And then finally he stands up. And he says, “I was wrong about whether a hot dog was a sandwich.” And everybody laughed and laughed. And I feel like it’s a really tiny way to admit that you’re wrong. But for me to admit times when I’m wrong, and then to celebrate students when they are willing to change their mind, even about something silly, it’s not going to get people to be totally open to changing their mind about controversial topics by the end of 16 weeks, or whatever. But it’s like, a little opening into practicing that habit of being willing to say, “Yeah, I was wrong about that.”

John: That reminds me of a podcast I listened to recently, I think it was Planet Money, or Planet Money Indicator, where one of the issues they were discussing is whether a burrito is a sandwich or not, because in New York State that affects its taxable status. So, it was actually a major political issue.

Rebecca S.: That’s so funny. I’m actually having my critical thinking students debate whether hot dogs are sandwiches right now in my asynchronous class, and actually, there’s not enough friction at the moment. They’re all like, “Yes, the hot dog is a sandwich, because it’s meat between two pieces of bread.” And I’m like, “No, the whole point is for us to practice disagreeing with each other.” So maybe I can throw in the podcast, or I can find some article.

John: I’ll include a link to that in the show notes. You discuss vulnerability a few times, might that be a little bit risky for a younger female professor in terms of the known biases that exists in terms of student course evaluations?

Rebecca S.: I think it’s really important for me to say that, yeah, I’m marginalized in the sense that I’m young-ish, female, professor in philosophy in particular, where there’s not a lot of women. There’s also things that I can do that other faculty can’t do. I have a lot of privileges as well. And I’m white and certain personality things, I think I can, like, get away with things. Everybody’s different. I don’t think there’s any wrong way necessarily, I’m not saying everybody should do what I do at all. I don’t think that that’s true. But at the beginning I was talking about what felt authentic to me. I think I needed to find a way to be able to be myself in the classroom. And so I don’t think that I can give some sort of universal prescription that will work for everyone. And I do think it’s risky. But I also think that the risks are unavoidable. I was saying before, that we’re not going to eliminate sexism or racism, oppression in the classroom, because our classes are part of the world. And those things exist in the world. So while I think it’s risky, I also think that it’s just risky to exist, [LAUGHTER] and that we are vulnerable, whether or not we want to admit it. This actually comes from Judith Butler, but there’s not really like whether we are or aren’t vulnerable. It’s just how do we manage our vulnerability. And so I think it’s not even necessarily about being more vulnerable. It’s just a different way of managing the inevitable vulnerability of being human. And being a teacher is super vulnerable. Being students also, humans are vulnerable, and that’s beautiful and scary. And so I probably framed it in terms of more or less, but when I think about it, I don’t actually think it’s about more or less vulnerability. It’s like, how do we manage it? Do we acknowledge it? Do we not acknowledge it? And I don’t think necessarily that we need to acknowledge it all the time. And different people are going to have different ways of thinking about it. But I do think it’s important to acknowledge epistemic vulnerability to a certain extent, because I think that it’s true that no one knows everything. And I think it’s harmful to perpetuate an idea that there is some sort of place you’re going to get to where you don’t need to continue learning. So I do think that epistemic vulnerability is important to acknowledge; other kinds of vulnerability, I think, may or may not be, it depends on the person.

John: And I would think it would also help to nurture a growth mindset in students when you acknowledge this epistemic vulnerability, letting them know that that’s just a normal part of learning, that there are many things they don’t know, but they can get there.

Rebecca S.: I share with them sometimes my own experiences of writing a dissertation and how difficult it is to get critical feedback. I don’t think necessarily everyone needs to do that. But I think that, for me, opening up a discussion where if I’m about to give them feedback on their writing, talking about what it’s like for me to get feedback on my writing and how that can be hard. And here’s sort of what I do when I’m about to read comments on something. Sometimes I need to take a minute. So, there’s ways of sharing that depending on your comfort level, but I do really think that acknowledging our humanity can be a really good and powerful thing.

John: We know you teach in a community college with a very high teaching load. And you mentioned you have a baby in the background there. But you also are playing with a band and you’ve released your fourth album, how do you record with a band and create music while also being a full-time faculty member with a heavy teaching load?

Rebecca S.: I’ve no idea. [LAUGHTER] No, I, you know, don’t sleep… No. Well, to be honest, and talking about, like acknowledging humanity and vulnerability, the last year has been incredibly difficult with a new baby. And it has been really, really, really hard. I have done it somehow. But I don’t actually think that the last year has been my best year of teaching, having a new baby. And I think that acknowledging that that’s okay, I’m still doing the best that I can, and things are going to get better now, both like daycare, and like, whatever, having a lot of support. I mean, you don’t do these things alone, also, I think. And so, I mean, I’m really lucky to have my husband who largely works from home. And he’s takes care of the baby a lot of the time, and he’s also in the band, and then having the ability to get a babysitter sometimes. So there’s a lot of ways in which the way that it works is the result of luck, and privilege and support. But then also, I think that if you’re doing things that you really care about, and that you love, you try to find ways to make it work and you just find ways to be more efficient. And like with teaching, one of the things I’ve realized is my impulse to want to reinvent everything all the time is not always what serves students the best. And so thinking about what I want to do, because I’m really excited about some new idea, sometimes doing the thing that I’ve done a million times that I know works is actually better for students and for me. And so thinking about that, too, like, my approach to teaching was always like, “Oh my God, every semester, I’m gonna do something wild and crazy and completely different, every single time.” And it’s unsustainable for me, but it doesn’t actually serve students. So I still do my experiments, and I think I always will, but I think being more deliberate. Okay, I’m going to take on this one project, I’m going to redesign this one class…

Rebecca: …and not like ten?

Rebecca S.: and not all of them.

John: And not redesign everything in the class, which I think Rebecca and I also have a tendency to try to do.

Rebecca: Yes, I’m trying to be more sustainable, my new approach.

Rebecca S.: And I think that oftentimes, it is better for students, I think that sometimes it might not be exciting for me, but it’s their first time experiencing it. And so I think that’s important to keep in mind.

Rebecca: Or you rotate between the things that you’ve invented, so that you stay interested, they’re still well established.

John: And the second time you do things you often have learned from past attempts at doing them, and they often result in better learning outcomes for students.

Rebecca: So we always wrap up by asking what’s next?

Rebecca S.: Yeah, so I guess I’ve already talked a little bit about what’s next in terms of like this world building class I’m working on. I’m also writing a bit about my role-playing game for ethics. I’m also working on something more about cultivating playfulness. So I’m interested in thinking more about exactly what it is about playfulness that I think is so meaningful and important and how that can be serious play. So I’m really interested in thinking more about these connections between playfulness, creativity, and inclusivity.

Rebecca: That sounds really exciting.

John: It does, and are you working on your next album?

Rebecca S.: Not yet. It just came out June 10. So we take a long time to write songs. So expect another one in like four or five years. [LAUGHTER]

John: We were doing a little bit of research on your work, and I ended up spending a lot of that time listening to music while Rebecca was actually reading your blog posts. [LAUGHTER]

Rebecca S.: Nice. Awesome.

John: I enjoyed it, it was really nice. Thank you.

Rebecca S.: Thank you.

John: Well, thank you. It was really great talking to you.

Rebecca S.: Yeah, thank you so much for having me. This was awesome.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

246. Embedding Career Competencies

Students generally enter college to advance their employment prospects. In this episode, Jessica Kruger joins us to discuss how explicitly embedding career competencies in the curriculum can engage and motivate students. Jessica is a clinical assistant professor in the Department of Community Health and Health Behavior and is the Director of Teaching Innovation and Excellence at the University of Buffalo.

Transcript

Rebecca: Students generally enter college to advance their employment prospects. In this episode, we explore how explicitly embedding career competencies in the curriculum can engage and motivate students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist&hellip

John: &hellipand Rebecca Mushtare, a graphic designer&hellip

Rebecca: &hellipand features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Jessica Kruger. Jessica is a clinical assistant professor in the Department of Community Health and Health Behavior and is the Director of Teaching Innovation and Excellence at the University of Buffalo. Welcome back, Jessica.

Jessica: Happy to be back.

John: We just saw you at CIT.

Jessica: &hellipgreat conference.

John: It was nice seeing everyone back in person again. For me, and I think for Rebecca too, this was our first conference in person in at least a couple of years.

Rebecca: Yeah. Yes, it was. Refreshing.

John: Speaking of refreshing, our teas today are&hellip Jessica, are you drinking tea?

Jessica: Iced Tea.

Rebecca: &hellipthe best kind during the summer.

Jessica: Exactly.

Rebecca: I have some nice piping hot Ceylon tea again.

John: And I have spring cherry green tea.

Rebecca: Is that a new one for you, John? Or you just haven’t had it in a while?

John: I haven’t had it in a while.

Rebecca: That sounds good&hellip

John: It is very good.

Rebecca: &hellipif you like cherries, which I don’t. [LAUGHTER]

John: It’s spring cherry, it’s not just cherry, these are spring cherries.

Rebecca: &hellipthe best kind.

John: &hellipfrom the Republic of Tea&hellip actually from Harry and David. But it’s produced by the Republic of Tea for Harry and David.

Rebecca: So we’ve invited you here today to discuss your work in incorporating career readiness into the public health curriculum. Why should there be an increased focus on incorporating career readiness in our courses and degree programs?

Jessica: I like to tell people, it’s not just our job to teach students really cool things, it’s our job to help them get a job. And so ultimately, by incorporating career readiness skills, we’re equipping our students to go out into the world and get their first job. And really, in public health, there’s so much work to be done. And so if students aren’t ready to do that interview, or send out their resume, or even talk about their experiences that they’ve gained in the classroom, they’re not going to land that job.

John: Has it been hard to convince other faculty of the need to provide this career readiness for students?

Jessica: So myself and another faculty member have been incorporating the career competencies in our courses, we both teach a 200- and a 300- level public health course. And, in those, we are getting students anywhere from sophomore, juniors, or seniors. And I think it makes sense in our profession where many students go out and get jobs after their bachelor’s to have them start thinking about this early, even as early as their second year in our courses.

Rebecca: Can you talk a little bit about the kinds of careers that your students in your program pursue?

Jessica: Yes, so we have students working all over. And I’m really proud to say we have a student who’s actually working at a safe injection site in New York City, which is one of the first that has been opened. We have students working locally in health departments, working for hospital systems, and local nonprofits. And so our students are really going out there and doing what needs to be done, especially what we’ve seen over the past two years with the pandemic. We’d had so many students work in contract tracing, and also with local health departments, and it’s growing. Public health is becoming more recognized. And that’s why we need to continue to prepare our students for what’s next.

John: For those listeners who aren’t familiar with safe injection, could you talk about what that is?

Jessica: The first facility was opened in New York City. This allows for individuals who use injectable drugs to go into a place that is clean and monitored and inject safely. Someone is there to monitor them, provide them clean supplies, and even help them if there is an overdose. So this is a harm reduction technique that prevents deaths.

Rebecca: Can you talk about the ways that you have employed career readiness into your courses?

Jessica: We use the framework of the NACE Competencies and NACE stands for National Association of Colleges and Employers. And these are competencies that were set forth by actual companies, employers out there, saying what they actually want in a new graduate. So there are actually eight different areas, things from teamwork, leadership, ensuring that students are able to be critical thinkers, have career and self development, include equity and inclusion, and be proficient and technology. And so what I’ve done is incorporate first starting at the syllabus level, incorporating some verbiage saying: in this course, you’re going to learn career competencies, and we’re going to cover seven out of eight of these career competencies. Because I’m teaching a 200-level course, I’m not focusing on the skill of leadership, that’s a little bit higher level that I’m working at in this course. But in every assessment that I have in class, I have not only the objectives, why we’re doing it, but I include what competencies we’re working towards, and how they can talk about this in a job interview&hellip to a internship site. So it’s not just that you’re writing a paper, you’re working on those written communication skills, and you’re able to articulate that

John: How have students responded? I would imagine it would increase their motivation a bit when they see how directly applicable these skills will be for them.

Jessica: It all comes down to transparency, because the more transparent and applied students feel that their assessments are, the higher quality I tend to see their work. And so by telling them, you’re not just writing a paper for me to read, you’re writing a paper to practice this skill. And you’re also going to do a presentation, because in public health, you need to have excellent written and oral communication skills. And hey, you’re also going to make a poster on Canva, because you need to know how to use technology. And so by kind of stringing these competencies together, it allows students to see that what they’re doing is not just for a grade, but to help them build those skills.

Rebecca: Can you walk us through a specific example?

Jessica: Yeah, in this course, I have students do a variety of writing samples on different problems. So, in public health we’re very applied. In one of the papers, students are talking about how public health has been influenced by other areas&hellip philosophy, psychology&hellip and so they could just see it as a paper, a 1.5 page paper, or thinking about it from a career readiness standpoint, they’re learning how to write succinctly and to whatever audience&hellip so, in this case, a lay person learning about public health. And so in writing this, I include why they’re doing it. And when I’m giving the example of what I’m looking for in the assessment, I often have the students reflect on what they’re doing and why they’re doing it. I think it’s important for them to think about that, why am I writing this paper? Why are we doing this? &hellipto improve your skills, to work on that written communication. Another example that I think is probably maybe a little bit more applied here is students write a paper, they record a short pitch of this paper using Flipgrid. So they’re doing written communication, oral communication, and in that little flip grid that they’re doing, they have to dress professionally, practice that skill, and give feedback to peers. And so all of that together, which is seemingly not a too arduous of an assignment, is really hitting on a lot of those career skills.

John: Have other faculty in your department picked up on the use of expanded career readiness in their classes?

JESSICA :Yes, one other faculty member has joined me and currently we are studying the effects of this on our students. So what we have done is ask our students at the end of the semester to complete a short survey asking them about how they felt about including career readiness into their courses. And overwhelmingly, students are so appreciative that we’re thinking about this, preparing them for what’s next. And it also shows that we care, we’re not just there to be a sage on the stage. We’re there to help them get ready for that job, that grad school, or whatever else they choose.

Rebecca: Do you have other findings from your research so far? I know you’re early in this process.

Jessica: Another major takeaway from this research is students wish they had this much earlier. They wish that we started talking about it day one. And while we could have told them about the career design center on campus, they’re not forced to practice this in their courses. They’re not tying this to that NACE Competency framework. But really, when we think about it, career readiness is everyone’s job. In our University of Buffalo, we’re working to create a career ecosystem, meaning that faculty are really on the front lines of this. They’re the first people that students are going to go to for career advice. And not all faculty are equipped to have that conversation. But at least, at the very minimum, being able to direct them to the Career Design Center and other resources that are available through the Career Design Center, I think is key. One simple way that I’ve introduced students to the actual physical Career Design Center on campus is I have them go take a selfie. They have a great little selfie station, and that’s one of their first things that they do in my course. So I can start to put a face to a name. And they can also learn about why it’s important to have a professional selfie. So they can put on their digital profiles like LinkedIn.

Rebecca: I found in some of the classes that I teach, I’ve in the past done assignments where we do professional email communication related to the work that we’re doing, or reports that might be common to the discipline. And students have responded much more positively to those kinds of writing assignments, because they can see the practical application and can connect the very specific, like, I can see how I’m doing this skill, and it’s gonna result in me being able to do this other thing. We just don’t always articulate that when it maybe is a little more abstract, when it’s maybe a more traditional paper and how that might tie to the kinds of work that they might do as a professional.

Jessica: Most definitely. And I’ll say this really was not much effort, I did not change what I was doing in my courses. I really just added a sentence or two to each of my assessments, relating it back to the NACE Competencies, I also brought a little bit more awareness to what we were doing in class. When we would do group work, I’d say, “Oh, you know what? You’re working on teamwork. That’s great, because in public health, we work in teams, and this is also related to the NACE competencies.” So it doesn’t have to be like a capstone course where you’re deliberately working on resumes or other career competency. But I think weaving this in, adding to it, and really raising awareness about some of these skills can really help students go to that next level, or start thinking about what they need to do to build some more skills before they leave our universities.

John: You’ve already answered this partly by talking about teamwork and other skills that can be done in any class. But what about those instructors in a course on abstract algebra? How would you build in, say, career readiness into that, other than the types of things you’ve already mentioned?

Jessica: Well, I think teamwork and technology would be two very easy competencies to weave in to any sort of course. Also, if you’re doing advanced level math, you’re probably using critical thinking skills. And you’re probably talking about what your findings are. So you’re hitting communication, you may not have much on professionalism, or career and self-development, but still, you’re hitting five career competencies, and not having to add anything, but really just highlight what you’re already doing.

Rebecca: Were you surprised when you sat down and looked at your syllabus and your learning objectives and looked at the NACE competencies and put them side by side and how well they aligned?

Jessica: Yeah, and in fact, it’s interesting, because in public health, we have an accrediting body called CEPH. And our CEPH Competencies for preparing students at the baccalaureate level actually align perfectly with NACE. And so it made sense, once you sat it down to say, “Oh, I’m already doing a lot of this.” And yeah, it might take part of my class time the second day of class to bring in one of the career designers so they put a name to a face, but it’s not taking away any time from my instruction. And by adding the transparency in that connection. I think it’s created more of a caring environment for my students. They know I care about what happens to them after this course. And more students have come to me than ever, asking about what’s next, whether that’s grad school, or how to apply for that first job. And that’s really rewarding. Now, I know not all faculty want more meetings with students. We all have busy schedules. But it’s also great to build those connections, because those are the students who are going to continue to be connected, have mentorship and be successful.

John: And we’ve always been preparing students for their future lives and careers. But we haven’t always been that transparent about it. And it sounds like that’s a really good approach. For someone who wants to start building this into their courses, how would you suggest they get started?

Jessica: I think one of the first things is go have a conversation with someone in your career design center on campus, see what they’re offering and see how you can collaborate with them. I found that our Career Design Center at University of Buffalo has so many resources so that faculty can literally plug in modules on career development that are already created for them. It was also really enlightening to learn what it’s like to be a student, to go to the career design center. And so sitting down with someone and understanding some of the intakes that they do, some of the questions, and even some of the tools that they have, really helps give you an overview of everything that can be offered to students. So when that student comes to you and says, I’m thinking about this career in biostatistics&hellip Oh, great, I don’t actually know a lot about biostatistics, but you should go to the Career Center, because they have a great tool where you can see what your life would be like as a biostatistician. And so first learn about your career design center on campus.. Second, I think it’s important to start slow with any new thing that you’re doing in your course. It may be that you dip your toe in the water and just connect some of what you’re doing with career competencies. So when you have students work in teams, say, you know, teamwork’s important. This is actually what people care about when you get a job. And here’s how you might want to talk about your experience in teamwork, if you don’t currently do this outside of the classroom, and provide them some of those prompts. And then if you want to dive in, and really incorporate your NACE competencies, I think for most professions, in most disciplines, the alignment will be there. And it’s not a ton of effort to highlight that, especially if you start with just your assessments and maybe highlighting some of those and then moving towards other things by maybe adding some career readiness modules, or having your students go to your career design center, take a selfie, whatever they have to offer.

Rebecca: Sometimes it’s really exciting and nice to have some ideas of ways to connect with students that are just so straightforward. [LAUGHTER]

Jessica: I’ll also say, while our students have grown up in a digital age, I find that technology tends to be their weakest category when we think about the NACE competencies. And so if your field uses any technology, talking about how to be proficient in that, how to be a lifelong learner, and how sometimes it’s hard to learn something new, but you have to if you want to keep up into the field,

Rebecca: One of the things that I have been doing in my classes that students have appreciated is when there are things like free online conferences related to the discipline, assigning them as an assignment. And to do that, and to talk about what that experience is like and encourage them to connect with professionals in the field. And a lot more opportunities for that kind of an experience has been offered over the last [LAUGHTER] couple of years.

Jessica: That’s a great way to incorporate that&hellip talks, and even as you introduce a new topic, having them recognize that maybe after this class, you’re not going to be an expert, but here are ways that you can build this skill that you may need. And it might be going out and trying something new, it might be connecting with another center on campus. But recognizing there are resources there for you.

Rebecca: I think underscoring the idea that you need to continue learning in your field is something that students don’t always immediately recognize without us pointing out. They don’t recognize that one of the things they might want to ask in an interview are what the professional development opportunities are, or ways to grow as a professional in their first job.

Jessica: Most definitely, and how they want to do that and how they can identify those areas of growth. That’s something that’s not often transparent. We send them out into the world and say, “Great, you’ve got this degree.” But there may be many areas where they can become a little bit more proficient or dig a little bit deeper into a topic. I want to also highlight that I think that this is so important for our first generation and our URM students. In our undergraduate program, we have about 37% of our students being underrepresented minorities or first-generation college students. And if we don’t talk about this, no one else is going to talk about this. And by becoming someone to turn to about career readiness and about asking those questions of “What do I do for my first interview?” or “How do I prepare for this internship?” &hellipwe’re not going to be able to build that for those students because they might not have someone at home to turn to. As a first-generation college student myself, I found that no one was talking to me about this. And so I think it’s critical in higher education that we think about this as an equity and inclusion component within our curriculum.

Rebecca: That’s a really good point. I’m glad that you underscored that Jessica.

John: And one other thing that I think many people have been doing is bringing in guest speakers&hellip and you can bring in some recent graduates to talk a little bit about some of those pathways. I think we’ve all learned how easy it is to bring people in remotely to give presentations. We don’t have to physically bring them to campuses anymore.

Jessica: Yeah, I think that’s really powerful having career panels. And it’s great to connect with our graduates. I love when they say that what they’ve learned in the class actually helped prepare them for what’s next. We’re not just shooting from the hip here, we really talk to our graduates, understand where we need to focus and continue to improve our program as we continue to grow.

John: We always end with the question: “what’s next?”

Jessica: Ah&hellip summer is upon us. And so it is my time to write up all the papers that I’ve been sitting on over the semester. One of that is the paper on incorporating these competencies into our courses. And I look forward to sharing the results when we get that out there. But I’m really interested in how we work with faculty to help them think about adding career competencies to their courses. Because I don’t think this just has to happen at the undergraduate level. I think the graduate level is also key. I teach both grad and undergrad courses. And as I’m revitalizing our graduate capstone, I really think that this is perfectly aligned, along with our competencies for our accrediting body. And so really, it’s all come together for me, and I’m really excited to see what other folks think about it as the word begins to spread.

Rebecca: Sounds like a really exciting opportunity. And I think you’re right, graduate school is a great place for some of these conversations to be happening.

John: Well, thank you. These are things I think we should all keep in mind, because even if we’re not thinking about career competency as being important for our students, students certainly are. It’s always great talking to you.

Jessica: Thank you very much.

Rebecca: Thanks, Jessica.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

176. Critical Thinking

“I want my students to think more critically” is a familiar statement in higher education, especially when we mix in conspiracy theories, pseudoscience and fake news. In this episode, Dr. Linda Nilson joins us to discuss practical techniques faculty can use to help students develop the skills necessary to become critical thinkers. Now Director Emeritus, Linda was the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Infusing Critical Thinking Into Your Course: A Concrete, Practical Guide as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

Transcript

John: “I want my students to think more critically” is a familiar statement in higher education, especially in a social media environment filled with conspiracy theories, pseudoscience, and fake news. In this episode, we discuss practical techniques that faculty can use to help students develop the skills necessary to become critical thinkers.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Linda Nielson. Now Director Emeritus, Linda was the Founding Director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Infusing Critical Thinking into your Course: a Concrete, Practical Guide, as well as many other superb books, book chapters and articles on teaching and learning. Welcome back, Linda.

Linda: Thank you very much for having me back, John.

Rebecca: Today’s teas are: …Linda, are you drinking tea?

Linda: I certainly am. And this is Twinings Berry, multiple berries.

Rebecca: Oh, yum.

Linda: Yeah. It’s delish. And I’m so glad you’ve helped to get me back into tea. [LAUGHTER]

Rebecca: As everybody should be,

Linda: Yes, yes, yes. It’s a wonderful thing.

John: I had some of the Twinings mixed berries early today. But I’ve got a chocolate mint black tea, which is really good.

Linda: Ooh, chocolate.

Rebecca: Chocolate anytime of day is excellent.

Linda: Yeah, really.

Rebecca: It’s an important food group. [LAUGHTER] Just like tea. I’m drinking that palm court blend that I have recently started making pots of. It’s pretty decent.

Linda: Great.

John: We just saw a note that you have this forthcoming book on critical thinking. What motivated you to write this book, now that everyone has this universal agreement on facts and reasoning and logic…

Linda: Yeah. Oh, yeah.

John: …and we don’t have a lot of disputes on these things anymore? [LAUGHTER}

Linda: Ha hah… Ah, yes. Actually, politics had nothing to do with it, if you can believe that. Originally, I got interested in critical thinking in the 90s. I decided to give a talk on it for some conference. And I got into the literature. And I said, “Okay, as soon as I get this talk over with, I am never, ever, ever going to do anything having to do with critical thinking, again, because the literature, it’s just a siloed mess.” So anyway, I just put that away. Well, then I guess it was something like eight years ago or so, Clemson University, in its infinite wisdom, decided to select critical thinking for its QEP, its Quality Enhancement Plan. And I thought, “Oh, no, do you know what you’re doing? Do you have any idea how difficult this is going to be?” And of course, I figured, like, I’m going to be doing workshops, so I got to get back into this literature. And I realized what I had to do was synthesize it for the faculty, because why should I inflict a literature’s flaws and warts on them. So that’s what I decided to do. Well, then, as it turned out, Clemson wasn’t the only university picking critical thinking for a QEP, and so I started getting invitations to do workshops, so I got on the road, and that’s always fun to do. I was back into it, but trying to make it as practical and as easy to implement for faculty as possible. So that was my goal, is always to make the faculty member’s job easier. So, I was comfortable with that. Well, a couple of years ago, David Brightman from Stylus contacted me, asking me to write a book on critical thinking. And I thought, “Well, I’m kind of enjoying being semi-lazy and just traveling around, oh, what fun.” And then I thought, “Well, look, I don’t know about this. But let me just start outlining the book and see how it feels.” Well, it felt really good. I was amazed. [LAUGHTER] And I thought, “Okay, lazy is not so interesting to me anymore. I’m going to write this book.” Well, he was happy. I was happy. And so I immediately started writing it, and because I had these workshops to work off of… updated by reading of the literature, of course… but other than that, it was probably the easiest book I’ve ever written. And so, this is fun, this is neat, this is great. And so I finished it really very quickly. And I was glad to do it. I finished it in four months or so… four to five months, the whole….

Rebecca: Wow..

Linda: Yeah, I know. Well, I was a house afire and I had nothing better to do. [LAUGHTER] So anyway, that’s how I was inspired to do it. It was really David Brightman ‘s idea. And if he was excited about it, that kind of got me excited about it. And I knew the need was out there. I knew the need because there were still universities crazy enough to adapt critical thinking as a QEP. I had also acquired some friends in the critical thinking community by then and so “Okay, this is fine. This is great. I’m going to do this.” And so I did it. And so now it’s due out March, maybe April, because you never know about these things. So it’s really happened. [LAUGHTER]

Rebecca: I found that a lot of faculty really believe in the idea of critical thinking, but have no idea how to define it or explain it to students.

Linda: Yes. And that’s a perfectly reasonable reaction for them to have, because of the massive literature out there. And it’s just the idea of critical thinking, it’s such an abstract, squishy, ephemeral idea. And we’re like, “What is it?” …and of course all faculty think they’re teaching it, right? But they’re not ,unless you make critical thinking, an outcome, a specific outcome that makes sense in your course. And you tell your students about it, and you call it what it is, that you’re not teaching critical thinking. It’s not going to happen by osmosis. It’s not going to happen by happy accident. Critical Thinking just doesn’t do that, it has to be a very, very conscious effort. And there’s a lot of literature to back this up.

Rebecca: So what is it? [LAUGHTER]

Linda: Oh, that’s a good follow up question. Well, I’m not going to go through the litany of definitions that you can find out there in the literature. But what I was able to glean from that literature is this: it’s interpretation, analysis, and/or evaluation, for the purpose of making some sort of decision or solving some sort of problem. And that’s it. But you could get definitions out there that go on for a couple of paragraphs, you can get definitions that seem to change color from one chapter to the next. I mean, they’re all there. And I just found it very confusing myself. But again, it was all about synthesis to me… all about synthesis, taking the best from these different silos, these different frameworks, and try to put them together into something that’s easiest, into a pocket definition: interpretation, analysis, evaluation. Now, you can put that in your pocket. And so can students.

Rebecca: I’m sold. [LAUGHTER]

Linda: Alright. That’s wonderful. That’s wonderful.

John: I remember when we introduced a critical thinking Gen Ed requirement here at Oswego, that everyone agreed that that was something that was desperately needed. It’s just everybody had a different idea of what it was. [LAUGHTER] And so eventually, it became infused within the disciplines. But I still think that that discussion of what exactly it is was never really resolved. It was kind of a way of sublimating that whole discussion and debate. So is critical thinking the same across disciplines? Or is it really going to be different depending on your disciplinary context or lens?

Linda: Okay, well, when you’re talking about interpretation, analysis, evaluation, yeah, that goes across the disciplines. But that’s not of much help to faculty. And so I put it in the disciplines, or I should say, in disciplinary clusters, because let’s face it, chemistry shares a lot with biology, right? …just in terms of general approach to observation, its approach to testing, just the general scientific thinking. And the humanities, there are a lot in common. And by the way, sometimes the social sciences pretend to be humanities, when you’re talking about the theories and things like that. Now, the social sciences, I look at them as sciences. So I put them in that scientific cluster. And then they’re the arts. Well, they’re off somewhere else entirely. They’ve got all….

Rebecca: Hey! [LAUGHTER]

Linda: Ah… oh, I’m sorry. You’re in it. But the arts are in a beautiful place. And they’re far more beautiful than the other disciplines. And then there are the, what I call the Applied Sciences, they’re different still. Oh yes, they share some overlap with the sciences. But so what I do is, there’s one chapter where I talk all about critical thinking outcomes by these disciplinary clusters. And now I have a list of different outcomes for each cluster. So take your pick, add more, but it’ll sound familiar to faculty. It’s using their vocabulary, it’s addressing their concerns, the sort of things that they strive to teach, the sort of things that they want their students to be able to do. And so I like to put it in the, at least, disciplinary clusters. Now it’s still you know, yes, you have to bring it down to your particular discipline, and then your particular course. But other than that, there are all kinds of tremendous overlap. But it’s such a different context. Sometimes you’re not even using the same verbs, and you’re certainly not using the same direct objects. I can assure you of that.

Rebecca: Why do faculty think it’s so hard to teach?

Linda: Oh, well, they’ve sure got good reasons. And I have a whole chapter on 10 reasons why teaching critical thinking is so challenging, and it’s got all kinds of reasons. First, there’s the literature, okay, that’s reason zero, okay. And so, I’m sort of, in a sense, telling faculty “Don’t read the literature, just don’t worry about that, or pick one and have fun” … whatever, but that’s not going to help you a lot. And that’s what’s different about my book, that mine’s, again, practical, concrete… here’s what you do… connect the dots. But reason number one is critical thinking about certain kinds of thinking, like that definition I gave you, but also a certain kind of subject matter. And there are courses out there that don’t have that subject matter. There has to be, in this subject matter, it has to be content containing claims, statements that may or may not be complete, valid, or at least the most valid, or the most viable, and for perfectly good reasons as well. For instance, there might be other respectable competing claims, the evidence supporting a claim may be weak or ambiguous, the data may be suspect, the source may have a lack of legitimacy or conflict of interest. So anyway, there are some courses that just seem to just be teaching undisputed facts. Now, if you’re teaching disputed facts, that’s fine, then you could talk about critical thinking. But there has to be some sort of dispute, some sort of competition, something among different claims. And again, there are, unfortunately, some courses that don’t have that… now you can add them in. Another thing about teaching critical thinking… It’s difficult for people to do, for students to do and it’s unnatural for students, for learners, for people to do, because we all want our current beliefs and values and ways of thinking confirmed. That’s what we are usually going for. And then there’s the whole struggle of learning. Learning is struggle, learning is effort. And students don’t seem to accept that. It’s like we’re supposed to make learning so easy. As easy as it was in elementary and high school. Isn’t that our job? …to make learning easy and effortless? No. But in any case, they might fight that: “Well, this is hard. So you’re not a good teacher. And anyway, students also bring biases and misconceptions into the classes that interfere with critical thinking. They’re not aware of these, and so you have to make them aware of them. And students don’t necessarily like that. Critical thinking requires self regulation, or metacognition. If you’re not observing your thinking, there’s no way you can think critically, I’m sorry, you have to question yourself all the time. But you’ve got to be aware of what that thinking is to even talk about it. Here’s another one and it’s nasty. Critical thinking requires the traits of good character. So virtues like integrity and determination and morality and inner strength. And some of our students just don’t have those virtues. Now, there are ways, there really are ways, that you can teach them about good character and bad character. And just the way good character is portrayed in our media and our literature, in movies, television, whatever, it makes good character look good, makes it look more attractive. But students don’t necessarily know what even good character is. And they can’t do self-regulated learning without having some intellectual virtues, like perseverance, like wanting to pursue truth, intellectual integrity, intellectual humility, certain dispositions that are absolutely essential. Otherwise, why bother to do critical thinking? Why put yourself through this? Here’s another thing that critical thinking requires: mental health, emotional health. Now, on the level that you can start increasing students’ mental health is on the level of defense mechanisms. But if you’ve got students out there, and I bet you do, who have personality disorders, they’re narcissists, they;re sociopaths, they’re psychopaths, so they might murder you someday. Forget about them, don’t even have hope for them. They have no interest in pursuing truth at all. That’s why they don’t wind up in the psychiatrist’s office. “What’s wrong?” “Nothing’s wrong,” right? And so what’s wrong with my thinking? Nothing’s wrong with my thinking. So they’re not into this, forget about them. Critical Thinking also requires students to respond to questions. And students don’t always like to respond to questions. They don’t mind telling you what day of the week it is. But beyond that, if they’re going to have to think, they might not be eager to do this. Now, if everybody else is doing it in the class, that helps. It really does. So yeah, there are all kinds of reasons why critical thinking is very difficult to teach, but it’s not impossible, and it’s not mystical.

John: You mentioned self regulation, what are some techniques that we can use in our classes to help students become more self regulated in terms of developing critical thinking skills that let them analyze evidence more effectively and interpret arguments more effectively?

Linda: Yes, well, again, awareness is where you start. So the first thing you want to do, the first habit that you want to develop in your class is whenever you have students give a statement… I don’t mean an undisputed definition, okay, I mean something that’s addressing a claim or making a claim in some way? And you’ve got to just keep asking them: “Well, how did you arrive at that response?” Now, you’re going to sound like a broken record the first couple of weeks, and then students will realize, “Okay, I’m not going to talk unless I have an answer to that question, because I know that Professor so and so is going to ask me this question.” So they start to think about this on their own, and they’re ready to answer it, or they’ll keep their mouth shut. But it’s not so impossible to look into your head as to how you came up with a certain answer. And initially, students will have to say, like, “I don’t know,” and you know what, that’s okay. Because now you have their attention. If they don’t know that that’s a perfectly fine answer, you need to start thinking about that, you need to start observing your thinking. And that’s the first thing you get them to do. Now, in terms of like, with readings, or if you’re having them watch videos and these sorts of things, you can give them reflective assignments to accompany those readings, and for that matter accompany any other kind of assignment. But what you want them to do is you want them to reflect on perhaps their affective reactions to the reading, or what they found particularly important, or what they found particularly surprising when they do a reading and then they have to answer this question or couple of questions at the end. And again, there are no wrong answers. This is like no stress, folks, just I want at least 100 words from you, that students start looking over what they just read and thinking about it from their own viewpoint. What did they find most important? How did they react emotionally? What connections did they make to what they already knew. So it’s making them more aware of what kind of impact that reading had. And those are just the simplest things in the world with substantial assignments, writing assignments, what you want your students to do is sort of a meta assignment at the same time. And again, you can’t go wrong, folks. And all you’ve got to do as a faculty member is just to check these in, just make sure that they did it and if you want 200 words, perhaps, “Okay, describe your reasoning in solving this problem.” or ”How did you reason through this case, in debriefing this case? What questions did you ask yourself along the way? What skills did you improve in the course of doing this assignment?” There’s so many different possibilities. And so, yeah, self awareness, self awareness. And then, of course, after a test when they get a graded exam back, make them aware of how they prepared or how they didn’t prepare. And so it’s making them face the music about, “Okay, what did you expect to get on this exam? And what did you get?” Gee whiz, how do you feel about that? Now, “A” students are gonna “Oh, yeah, I thought I got an A,” or I thought I’d get a “B” but I got an “A,” and so I feel whoopee.” Okay, don’t worry about them. They’re already self regulating, they self regulate to the point of paranoia, they’re the least of your worries. [LAUGHTER] But other students need to become aware of, “Okay, so how did you study? Did you study the way I told you too, which is quizzing yourself? Or do you just reread, reread, reread, reread the way I told you not to, because that’s a waste of time? How many hours did you study? Was this enough? What are you going to do differently on the next exam to do better?” And so students have to think about a strategy, they’re meta-studying in a way. And so they’re thinking about their learning strategies. And they really need to do a lot more of that. Any kinds of like, experiential activities… there are all kinds of questions that they can ask themselves, or you can ask them to write about, “Well, what were your goals originally? And how did they change through the simulation or the game or the role playing?” And this would be true of like service learning or field work or any of these things. “How did you respond to other people involved in the interactions? And why would you respond differently next time?” So there’s a lot of self assessment going on. And this is a good place to start with self assessment. It’s a safe place to start with self assessment, because usually everybody changes their strategy from the beginning through the middle to the end. And that’s good. That’s what they’re supposed to be doing. That’s called learning. [LAUGHTER] But it makes them aware of that learning. That’s how you do it. Yeah.

John: It sounds like there’s quite a bit of overlap between improving students’ critical thinking skills and improving students’ metacognition,

Linda: Absolutely.

John: Is it effectively the same type of strategy used in both cases? Or is it essentially the same process In both cases?

Linda: What’s different about it is with critical thinking, you’re dealing with real disciplinary content. And you’re not necessarily doing that with metacognition. Yes, self awareness is definitely a part of critical thinking. But that’s not all. I mean, you’re actually trying to come to some sort of conclusion or solve some sort of problem, maybe even decide to take some sort of action. You’re examining claims, you’re examining data, you’re examining sources, you’re examining a lot of different things along the way that you’re not necessarily doing in metacognition, but metacognition, it precedes critical thinking. I mean, if you’re not aware of how you’re thinking, how in the world, are you going to come up with a well reasoned approach to a problem? So there’s a lot more. You’re using metacognition, to extend reasoning into something that is disciplinary based.

Rebecca: I’m so glad we’re having this conversation today. Because I’m working on a new class where practicing thinking moving into higher-level classes is the goal of the class. And so I was working on a lot of reflection assignments. So I was just doing a little editing while we were chatting. [LAUGHTER]

Linda: Oh. alright, See, I’m glad this is useful, and that’s the whole thing. I want that book to be useful. And that’s what makes it, I think, different from the other books out there and critical thinking, because most of them are in a particular silo, they’re coming from a particular silo, they’re may be extending that silo in some way. And I was looking for common ground across multiple silos, and then maybe not all of them, but multiple silos, I tell you one thing that they all have in common, they all say that critical thinking involves metacognition or self regulation.

John: And I remember reading a really good book on creating self-regulated learning.

Linda: Oh yeah, that one, yeah.

John: That might be something we’d recommend to faculty.

Linda: Yes.

John: We’ll include that in the show notes as well.

Linda: Okay, good. Yes, indeed, I’m glad I wrote that book beforehand. I really am. [LAUGHTER]

John: And it ties in nicely with a lot of your discussion in the book. In this new book, you’re also addressing how specifications grading could be used. And I know there’s a really nice book on that as well.

Linda: Oh yeah, yeah.I remember that one.

John: You mentioned that when you grade things, that it doesn’t necessarily require a lot of work for the faculty member. Could you just talk a little bit about how specifications grading could be used to evaluate these types of activities?

Linda: Sure, absolutely. First of all, all assignments, not courses, all assignments are graded pass/fail, and for that matter tests, but you don’t pass either a test or an assignment at a C level, you raise your expectations to a B level. But guess what, you’ve got to know what components in that essay or that paper or whatever that design that you want to see to achieve a B level. Now, if you already have rubrics, fine, you can start looking at the top level of the rubric and maybe take a few from the next one down. But those are the things that have to be in the piece of work for it to pass. And by the way, you explain this to students. Now the stakes suddenly become higher for students. So guess what? They read the directions? Isn’t it wonderful? They do, they want to pass. Now, initially, they don’t believe that they could possibly fail at anything because of partial credit. Well, guess what, there is no partial credit here. But they won’t believe it till they fail something. So you always want to give them some get out of jail free cards, so they can maybe fail a couple of times, and then redo the assignment the correct way, because they realize, “Oh, he or she is really serious about this. Oh, goodness. Okay.” So yeah, they really start reading the directions and doing what you asked them to do. And you know what? If they’re worried that they’re not doing it, they visit you, they call you, they email you, they actually ask for clarification, because this means something. And we’ve been lowering the stakes and lowering the stakes and lowering the stakes for years now, because we don’t want to cause them any stress. Well, you know what? We’re going to have to cause some stress. Learning is stressful, sorry. It’s just the way life is. So anyway, when you know what you want to see there, and this is where the thinking is… this thinking it might be before the course starts. So once the course starts, you’re on easy street, because all you have to do is look for those elements in the piece of work that the student hands in. Something missing? You mark which element is missing. If you want to make other comments, hey, far be it for me to tell you not to write comments. But you don’t have to, you don’t have to, and relatively short assignments. You don’t have to, unless you want to. I mean, if you’ve got a student, they had all the elements in that, you want to draw a happy face, knock yourself out. [LAUGHTER]

John: Or at least for the artists, among us. [LAUGHTER]

Linda: Exactly. Now, if you want to go specs grading… because we’re talking about specs, we’re talking about specifications, like in a computer program, essentially… but you really have to look in the mirror and say, “What do I want my students to be able to demonstrate that they can do in this assignment, or on this test, or in this particular essay?” And that can be hard, because we’ve never had to be so specific, but this is what students love about specs grading. We get specific, we tell them what we want, they know what our expectations are, and they haven’t known, this might be the first time they’ve ever really known, what we wanted, because we have to be specific. And if we left out a specs that we wish we’d put in, well, better luck next semester, right? Just remember to put those in. But you can’t change those once you give students your list of specs. And then there’s bundling. With specs grading, you can also bundle assignments and tests together for various grades where students decide, “Oh, you know, you have to do so much work for an A, and it looks very difficult. I’m not going to go for an A. How about if I go for a B and that way this isn’t that important of a course to me. It’s alright.” And they might decide to go for a C. And you know what? what should you care? This is their decision, and if they’re going to be happy with a C, you can respect that. And otherwise, we look at C students like, well they were lazy and didn’t care and this and that. No, no, no. Students choose it for themselves. Fine. No problem. We don’t look down on our C students, as we unfortunately will kind of do sometimes. Anyway, but that specs grading. We’re on critical thinking and critical thinking, if you have the specs for something written, or a presentation for that matter, but if you know what you want students to show that they can do in terms of critical thinking, this is an easy way to grade, it really is. All the work is upfront.

Rebecca: Yep. It’s very time consuming upfront with what I’m doing right now. [LAUGHTER]

Linda: And it’s stressful. It’s stressful for us, because we “Oh, should I give this five points or six points?” Forget it. Don’t worry about that. And you know, students will go to you and argue for a half a point, they don’t mess with that with specs grading at all. They don’t do that. So it saves you a lot of wear and tear. And it’s better for your relationship with your students.

John: In large classes, how can you help students develop critical thinking skills if you don’t want to read lots of written work?

Linda: Ah, okay. Well, this is the beauty of groups. There are two methods that are particularly important in teaching critical thinking, there’s discussion, and that includes, by the way, debate…. debate’s really good, but discussion, debate. And you might have to do that in terms of just teaching in small groups. And that’s okay. Because the whole idea is that students are getting different points of view on a claim or a statement or something like that and then they have to defend it. But that’s all something that you can have students do in groups, those sorts of activities. Another thing that’s very important are some kinds of problems to solve. And I don’t mean the cookbook types. I’m talking about complex problems, fuzzy problems, if you will. And again, usually we have students work in groups for that, anyway. So that’s how you can teach it in large classes. It’s lovely if you have a small class, and everybody can hear everybody else’s statements and discussions and things like that. But if you don’t have that luxury, you can still teach critical thinking in large classes. And again, this is where spec grading can really, really shine. Because your grading time is cut to a fraction, an absolute fraction. Let me say one other thing. I talked about methods, but really the key to teaching critical thinking is questions, the questions that you ask your students, whether it be in class, or whether it be a writing assignment, or a paper or for a design or presentations, whatever it is, the key is questions. And each silo, each framework offers certain questions, that here you should have your students do this or answer these questions. So what I did was I put all of those together, synthesized them into a list of 45 critical thinking questions. Take your pick. Now, of course, you can look at your outcomes, and they make excellent assignments and questions and things like that. But these are general questions that make students think critically. And of course, you always follow up with “Well, gee, how did you come to that response? How did you arrive at that?” So anyway, there are certain questions that guarantee students are doing critical thinking and you have to adapt it to your particular class, but a lot of them, they go across the disciplines. And this is where you can see some of the similarities, like asking about the source of a claim, for instance, can you trust this source?”

Rebecca: How do we help students that come in with biases, preconceptions and things?

Linda: Oh, yes, absolutely.

Rebecca: …resisting facts. How do we start to overcome that?

Linda: Well, I think what you need to do is you start your course… you might have to start your course anyway… with selling critical thinking to your students. Because first of all, critical thinking is a skill that employers want. And I start my book with that, just so much evidence of that, say analytical thinking and that’s the same sort of thing. But there’s also beyond employment, there’s a quality of life. Now, if your students do not want to be suckers and fools, and if they don’t learn to think critically, that’s exactly what they’re going to be, suckers and fools. For instance, it takes critical thinking to avoid scams and shams. And if you google “scams,” you get billions of hits. It’s horrifying. Questioning and other things like they don’t know how to make sense out of popularized research studies. Well, a critical thinker knows enough to check those things. Advertising, that’s another place where you have to be very discriminating. And you have to critically think about “Well, what did that mean that such and such is twice as effective? Twice as effective as what? And what does ‘twice as effective’ mean?” There’s propaganda, fake news, disinformation, demagoguery, doublespeak, all this stuff to cover up the truth. And what students don’t know… a lot of people just don’t know… is there are fact-checking sites out there, there are quite a few of them. I talked about eight of them in my book, and how to get there and where it’s from, who puts this together. But it’s a good idea, I would say to give students an early exercise. And it could be, if you’re teaching chemistry, make it chemistry, if you’re teaching political science, make it political science, politics, to give them some fact-checking assignments. And if you’re worried about the politics, like all of these sites will call both sides of the political debate down on their lies or their distortion. So they’re fair that way. And so, if you want to get students to question their own politics, it’s great if you can get some good examples of things that they can go research where there’re lies on both sides, and that shows them that. But then they start realizing, “Oh, my god, there are lies. Oh, goodness, I’ve been believing all this.” There is Politifact.com. There is FactCheck. There is FlackCheck. There is OpenSecrets, Media Bias / Fact Check. There’s apnews.com/apfactcheck. There’s Snopes. There’s Truthorfiction.com. I mean, there’re all these sites. There’s so many of them, and that’s just the few that I talked about, because they tend to be better known. And they’re put out by like AP or something or Annenberg Foundation. And so that will teach them how much is garbage out there. Again, from both sides. Both sides in the political debate are full of garbage. But you also have debates going on in biology or in medicine. Remember, I’m sure you do, when they were saying, “Oh, coffee is terribly bad for you.” And then three years later, “Oh, it’s very good for you.” So it really hits all fields. Some of my very favorite ones is doublespeak. And I don’t know if students are aware of doublespeak like “servicing the target.” That’s military talk for bombing, to service a target. Yeah. “Neutralize” that means to kill. “Downsizing” means firing employees, “misconduct,” white-collar crimes committed by politicians, business leaders, military professionals, the police, that’s misconduct versus crime. “Detainee” is a prisoner of war. I mean, I could go on and on and on. And I do in my book.

John: You mentioned how important critical thinking skills are to employers. What does the evidence show about how effective colleges are in terms of helping students improve their critical thinking skills?

Linda: we’re not so hot, but it’s just put it that way. According to Academically Adrift, we improve the critical thinking skills of about one third of our students. Now, I don’t think that’s anything to celebrate myself. But I mean, at least we do that. But, these are often the students who came in with some critical thinking skills to begin with. The ones who need the help the most to critically think are not getting that help. They’re just not getting it. And so we’ve got to go out of our way to reach them. Again, most faculty think they’re teaching critical thinking when they’re not because they’re not consciously doing it. So there’s a sense in which we haven’t even scratched the surface of what we can teach in terms of critical thinking. And again, with all the different definitions out there, and all the different silos, I don’t blame faculty at all.

John: We can hope that your book will help reach more faculty and help us be more effective in this task.

Linda: Let me add one other thing that you can sell students on. Critical thinkers, and there’s research on this, tend to experience fewer difficulties in life than others do, whether they be health difficulties, financial, personal, legal, they just lead less troublesome lives. So if you kind of want to be happy, it’s a good idea to think critically, because it’ll keep you out of difficulty.

Rebecca: A good selling point, a good selling point.

Linda: Yes, it is a good selling point. [LAUGHTER] It is. And that’s just it. You might have to sell students on it. And you’re going to have to explain what critical thinking is to students, but they can take that pocket definition. It’s not negative thinking. It’s not just criticizing things. It’s nothing like that at all. And it’s not leftist, that’s not it at all. But they need that clarification because they’re coming in with misconceptions about what critical thinking is. And what the words sound like. Maybe it’s not a really good name for it, but that’s what we’ve been calling it. So we’re stuck with it.

Rebecca: It also seems like faculty make a lot of assumptions about students knowing how to analyze or how to interpret and what the question focus that you’ve indicated we should pay attention to, is really getting at that by asking the questions to get them to do the activity that we want them to do, when they might not know what that word actually means. Because it’s not been modeled.

Linda: Exactly, we should stop and model and then say, “You know, this is an analysis question that I’m asking you or that I just asked you.” We need to label things, so students get to understand what these cognitive operations are. And unfortunately, we make assumptions and studentsmight say, “Oh, analysis, that something a shrink does.” No, no, no, no, no, no. Interpretation. “Well, that’s something that people who read novels do.” No, we are all doing it all the time. So even chemists do interpretation. You interpret the data, and you can come up with different things.

Rebecca: And a lot of students get stuck right on describe, they don’t go much past that, and they don’t know that that’s what they’re doing

Linda: …or summarize. Yeah.

Rebecca: I’ve had conversations with students about those kinds of words before and it was really productive.

Linda: Yeah, yeah. because nobody’s really told them what these words mean.

John: It seems that higher ed is not doing quite as well as we’dlike in terms of increasing students critical thinking skills. Should this be an important focus for higher ed?

Linda: First of all, the buck stops with us. Otherwise, they go out into the world. And they could be adrift in general, without our teaching them critical thinking. They’re not likely to learn this in K through 12. I’m sorry. Oh, yes, they will in prep schools, but we don’t worry about those kids. So we have to keep people who are in power, and people with money honest, and the only people who really have ever wound up doing that are fairly well educated people. That’s why higher education is so important. And unfortunately, over the years… certainly, from when I was in college to now… the whole reason people go to college, and what they want out of college has changed. Now they want a job. Well, I hate to use this phrase. Back in my day, we wanted to develop a sound, sophisticated philosophy of life. Now that fits right in with critical thinking, critical thinking was very much a part of it. But now, that’s kind of like out the window. Well, to keep a job, you need to become a critical thinker. This is what employers need. This is what the world needs. And we can’t do good production without it. But even more important than that, we talk about freedom, for this society, any society, our freedom lies in our awareness of our patterns of human cognition, how they can be exploited, these patterns. And just, in general, our awareness of this worldwide pandemic of dishonesty in the pursuit of money and power, and it’s almost considered okay, because it’s all around us, we see it. Well guess what? It’s not okay unless we say it’s okay. So if we want to preserve freedom, we absolutely need this awareness. And it’s not like we’re doing something that hasn’t been done before. Because really, this awareness and this watchdog-type thinking really gave rise to a lot of successful movements, the environmental protection movement, the conservation movement, sustainability, recycling, stronger vehicle safety standards, and that’s just to name a few. And they all came out of critical thinking and awareness of where our weaknesses as human beings are. We all have a mind… well, almost all of us have a mind… and we need to know how it works so we can protect ourselves from all the advertising and scams and shams of demagoguery… which by the way, appeals to emotions… to all these different trips that individual institutions, organizations, corporations are trying to lay on us. And they are, because, hey, its profitable to lay the sawdust if we buy into it, we can’t afford to. We should not.

John: We always end with the question: What’s next?

Linda: Ah, yes. What’s next? Well, first of all, I hope getting back to travel.

Rebecca: Me too.

JON: Me too.

Linda: Yeah, exactly, exactly. Doing workshops and keynotes instead of this virtual nonsense where you’re talking into a screen. To see an audience tells me everything about what I need to do next in the workshop or in the keynote. But aside from that, I’m just finishing up another book: Online Teaching at its Best, the second edition. And in the second edition I talk about the kinds of teaching we’ve been doing lately, like remote teaching, hyflex teaching, Well, hybrid, we’ve been doing, but talking about it in another way. And in addition to the fully online learning that we talked about… Ludie Goodson and I talked about in the first edition… but the second edition has a lot more about what we’ve been doing lately because it’s possible universities and colleges might decide to keep some of this remote stuff. Hyflex is very much a disaster when you’re dealing with this pandemic and masks and all that. But, anyway, universities might say, “Hey, you know, it’s cheaper to do remote, we don’t have to build any more buildings, students can learn at home, hey, students might like that.” But again, they might not. But I’m afraid they’re going to do this with conferences, and it just makes me want to cry. Let’s do virtual conferences. We don’t have to pay hotel bills. And people don’t have to worry about getting travel funding. But in any case, it worries me that we’re going to get too accustomed to and accepting of, this virtual communication, which we all know, deep inside of our hearts, that’s not as good as human face-to-face contact. So anyway, but that’s next, that book will be coming out, I think, in the summer. And right now Ludie and I have the manuscript in but we’re still doing some pre-production stuff having to do with author queries, proofs, all that nonsense that makes you wish you didn’t write the book. [LAUGHTER] Writing is the fun part. This isn’t. But I’ll do it. I promise.

John: More information on effective online teaching, I think, is especially important these days. And I think we’re gonna see a lot more people doing it even when the pandemic is over, now that people have learned that it can work.

Linda: Yeah, it can work. But we just started doing these things, not knowing how to make it work. And learning has suffered, grades have suffered, and rigor has suffered. So far, no real good has come out of the switch. It can, it can, and that’s what the second edition will hopefully begin to do.

Rebecca: Well, thank you so much for joining us again, and having such a rich conversation.

Linda: Well, thank you for this opportunity. This has been great. Thank you.

John: And our faculty very much appreciates all the work you did with ACUE as well, because they keep bringing up some of the things they learned watching some of your videos in that course.

Linda: Oh, good. I’m glad to hear that. Well, thank you ever so much for this chance to talk about a book that isn’t even out yet but that I’m truly in love with.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

110. Fostering a Growth Mindset

Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, Sarah Hanusch and John Myers join us to discuss how they have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY Oswego.

Show Notes

Transcript

John K.: Some students with fixed mindsets enter our classes expecting to be unsuccessful while others believe that they have a natural talent in the discipline. In either case, these students often get discouraged when they experience challenging tasks. In this episode, we examine how two faculty members have revised their classes and used metacognitive exercises to help students develop a growth mindset and to recognize the benefit of learning from mistakes.

[MUSIC]

John K.: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John K.: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Sarah Hanusch and John Myers. Sarah and John are both Assistant Professors in the Department of Mathematics at SUNY-Oswego. Welcome, John and welcome back, Sarah.

Sarah: Thank you.

John M.: Thank you.

John K.: Our teas today are?

Sarah: None today

John M.: Yeah, imaginary tea. No tea for me.

Rebecca: The imaginary tea…that’s what my daughter likes to drink. That kind.

John M.: Yeah, I’m in good company there&hellp;

Rebecca: I have English afternoon.

John K.: And I have a ginger tea.

Rebecca: We invited you here today to talk a little bit about how you’ve introduced a project on metacognition in some of your mathematics courses. Can you tell us a little bit about the project?

John M.: Sure, this began, I believe, in the spring of 2018 in a Calculus I course. And the idea was that, Calculus I is known across, basically the entire country…every school in the country…as being a very difficult course. So, you have a lot of students who are coming in, especially in the spring semester, who had bad experiences with calculus in the past. And in particular, I’ve been told by some colleagues that there’s going to be some students in there that more support than I suppose you would imagine. The situation was that on the very first day of class, I had students coming in who have had bad experiences with it in the past. And then at the same time, I have the students that are typically high performing. And they have difficult times also with perfection, you know, being obsessed with 4.0s and grades and that type of stuff. So the idea was that I wanted to simultaneously address failure with the students and perfection at the same time. And I was sort of led to think about this metacognition project, actually, funnily enough, on a flight back from San Diego. I was at what are called the joint meetings for mathematicians, and a lot of progressive newer teaching techniques are talked about at this conference. And I’m flying back from the conference on the airplane and I’m getting really introspective and I’m thinking like, I really need to do something to talk to my kids about failure and perfection. And then it occurred to me that there was this blog post that I had just read a couple weeks before by a mathematician by the name of Matt Boelkins at Grand Valley State University. And he had this idea for a metacognitive project that addressed all sorts of things like growth mindset, fixed mindset, productive failure, and all these different things. And I decided about a week before classes started that this is what I was going to do.

Rebecca: That’s when all the best ideas happen.

John M.: I know…right before class and on an airplane. I get really introspective when I’m on airplanes and staring out the window and thinking of all the big things in life and stuff.

Sarah: And essentially, John came to me and said, “I’m thinking about doing this project.” And I said “Well, that sounds cool. And let’s see if we can measure if it has any positive effect or not.” So, I sort of came in on the research side of it…of “let’s see if this is effective for changing attitudes towards mathematics.” And since then, I’ve stolen the project to use in my own classes. But, it really started as I came in sort of more on the research side of things

John M.: I think stolen might have been a strong word, but…

Sarah: I didn’t ask…I just took it. [LAUGHTER]

John K.: For the research project did you do pre- and post-tests on attitudes?

Sarah: We did a pre- and post-test, we use an assessment called MAPS which is the Mathematics Attitudes and Perceptions Survey. It’s a 31-item survey. It assesses, I think, it’s seven different dimensions. Some of them are growth mindset. Do they view mathematics as being answer focused or process focused? The categories were growth mindset, the applicability of mathematics to the real world, their confidence in mathematics, their interest in mathematics, their persistence in mathematics, their ability to make sense of mathematics, and do they view mathematics as being answer focused or process focused?

John K.: Sounds like a good instrument. Before we talk about the results, let’s talk a little bit more about how you implemented it. How was the project structured in terms of what activities did the students do during the class?

John M.: So the idea was that over the entire semester, they would have a selection of articles online to read, they would have a selection of YouTube videos to watch and it was essentially experts that are addressing these various topics. So, like for example, there is a clip by Carol Dweck, one of the originators of the theory of growth and fixed mindsets, and they were to watch these clips and read these articles across the semester. And then I think it was probably with two weeks or three weeks left in the semester, they’d have to write a reflective essay. It was an attempt to sort of shift the culture in the classroom towards viewing mistakes and failure as productive and as opportunities for learning. Because I think in wider culture, everybody believes that math is just about the right answer. And that if you can’t get the right answer, then there’s no worth in whatever effort it was that you put in to get to that point. And I wanted to provide sort of a counterpoint to that, so a counter narrative. Being honest about how many times per day mathematicians actually do fail, you know, that type of thing. So yeah, the main component was this essay that was reflecting on the stuff that they read and watched over the semester, and then there was sort of like daily conversations.

John K.: Were the conversations online or were they in class conversations?

John M.: In class…in office hours, just kind of whenever they popped up. I remember a couple conversations that happened after I gave back exams, for example, or rather right before I gave back exams. So for example, I would say, you know, I’m about to hand back exams. And I want you when you see the score, when you put the paper over and see your score, I want you to immediately think how are you going to frame this result in your mind. Are you going to look at that score and be happy with it and chalk it up to just your natural talents? Or are you going to say, “Oh, this is a result of hard work?” And then if you’re not happy with your score, are you going to put it away and never look at again, or are you going to engage with your mistakes and make them productive mistakes? It was sort of intervention through conversation that happened on an almost daily basis.

Rebecca: Did you notice a difference in the kinds of conversations you were having in class because they were doing these readings and watching these videos, maybe conversations you hadn’t experienced before in the classroom?

John M.: Yes. In particular, I had students come into office hours and they were relentless with trying to understand the material because they knew that they were going to have another shot to get it right. And I had never experienced that before. In fact, in one of my student’s essays, I had a student tell me that when she’s not done well on exams in the past, she would just take the exam and stuff it into her book bag and never look at it again. And she told me that just because of because of how I was structuring the course that she doesn’t do that anymore. She actually pulls it out and engages with the mistakes and the comments that I put on the exam and comes and talks to me about the exam and everything. So I did see a change in the students.

John K.: Was some of it based on the reflections or was it also partly based on a restructuring of a course to give students more opportunities to redo things or to try things again?

John M.: I believe the latter had something to do with it. Because the idea was that I could say these things out loud to them. But I wanted to actually build components into the course in addition to the essay that sort of reflect the themes that I’m trying to communicate to them.

John K.: Telling them that they can learn from mistakes, if you don’t give them the opportunity…

John M.: Right.

John K.: …to learn from mistakes might not be as productive. I think both components are really valuable. I just want to make sure we were clear on that, too.

John M.: I think that you risk sounding like a cliche motivational poster, if you don’t actually put some meat on the bones with it.

Rebecca: Can you talk about some ways that you actually built that into the course?

John M.: I did test corrections. I don’t remember exactly, I think it was get back half the credit they missed or something like that. So, the idea was that they had to engage with the mistakes on their exams and correct them. And it had to be perfect. So they had a week to turn in their test corrections, and then I would re-grade them. This was very time consuming, as you might imagine, but the students I believe, really responded to it. It really sort of hooked in with the theme that I was trying to send.

Sarah: And since then, we’ve both moved to more mastery based grading. John before I did, but a system where students keep trying things until they get it right. And that really helps sort of drive that “learn from your mistakes” message home.

John K.: Are you able to do some of that in an automated way? Or is this all involving more grading on your part?

Sarah: The way I’m doing it, unfortunately, it’s more grading on my part. Although I will say this semester I’m doing these mastery based quizzes, but I’m not collecting homework. So, it’s kind of a toss up in terms of how much…it isn’t really extra grading. I’m just grading more things in another category.

John M.: Right, I would not do test corrections again. Not only was it a lot of time to grade, but then I had issues with academic honesty. The mastery based thing I have found is, I believe, much more effective.

John K.: Another thing you may want to consider that we’ve talked about in a couple of past podcasts is having a two-stage exam, where in the first stage, they do it themselves. And then you have them break up into groups and do either all the questions or a subset of those as a group. So, you’ve got some peer instruction going on as well…and that way it’s done right in class and it can be done, if the exam is short enough or the class period is long enough you can do both of it. A common practice is to do two-thirds say individual and then one-third for the group activity, which has many of the same things. They don’t know what they’ve gotten wrong, but when they’re sharing with their peers, they’re talking it over and it means you only have to grade the group exams on the second stage, which makes it a whole lot easier than individual ones.

John M.: Right. Yeah, I have a friend I believe he has done that stuff like that. So yeah,

John K.: The Carl Wieman Science Education Institute, I believe, has a lot of information on that. I’ve been doing it the last couple of years, and it’s been working really well. Doug Mckee was a guest on an earlier podcast, we talked about that as well. Are there other things we want to talk about in terms of what you’ve done in the courses?

Sarah: One thing that we’ve both done since this initial project is we’ve taken some of the ideas of this project, but interspersed it more throughout the course. One thing I know at the time that John observed was that he felt like a lot of the students started the projects in the last week, right? And so what I’ve done instead of doing a big project of these topics is I’ve taken these articles and done the second week of class, you have to read one of them and respond on it. And then the fourth week, you have to do another one, and so on. So it’s a little bit of it throughout the whole course instead of all loaded at the end. I think it helps having some of those conversations with the students as well because they’re not just seeing the ideas in the conversations. They’re not just seeing the ideas in the paper. They’re kind of seeing both and it just helps intersperse it a little bit throughout the semester. I know I’ve done that a couple times now. I think you’ve done that since as well.

John M.: I did a pre-semester sort of essay and then I did a post-semester essay. But it was in response to the first time we did that, which is referred into the paper, and one of my students actually told me in their essay, he was like, ‘Hey, I wish I had this at the beginning of the semester.” So yeah, it’s definitely like a “duh” moment. Like, I probably should have done something earlier in the semester, instead of waiting all until the end. But, you learn as you do these things, so. But the essays that the students wrote… I provided them with prompts just to alleviate any sort of writer’s block that they may have. But, the students who basically ignored my prompts and told me their personal stories were the essays essentially that I still remember. I had students that were straight A students that were telling me exactly what I thought was going to happen: that they’ve been the smart person their entire life, and they kind of feel trapped by being a smart person. They don’t want to take any risks because if they risk something and fail, then that’s their identity as a smart person, right? They’re not smart anymore. I’ve had students from the other end of the grading spectrum who basically told me that the first day they walked into the class before I even said anything, they were already convinced that they were going to fail the class. I had students tell me about mental health problems. I had adult learners talking about balancing life and school issues. I mean, it’s just absolutely amazing what they told me, they opened up basically. That made a big impression on me.

John K.: Tying into an earlier podcast, Judie Littlejohn and I had introduced something really similar where we have weekly discussion forums. And I also noticed the same sort of thing, that I got to know the students much better because when they were talking about some of the barriers or the issues they face, they were sharing a lot of details about their life. And you get to know them better and they also seem to form a little bit more of a tighter classroom community because they also got to know each other a little bit more.

Rebecca: It is kind of interesting how when students are talking about their process or who they are as learners, is very different than talking about the subject matter. And it does get them to open up and may be engaged with faculty in a way that they wouldn’t otherwise.

John M.: And I have found being honest about my own failures in the past has been a catalyst for conversation, right? Because they view us as professors, they view us as the authority figures, the experts in that we never fail. And basically telling them how many times I fail on a daily basis in my own mathematical research. It goes a long way, I think… finding common ground with them. And acknowledging how difficult the subject material is. I mean, there’s a reason that calculus has a high failure rate because it’s a hard course, among other reasons. Yeah, just having the humility with the students and kind of stepping down off of the pedestal in front of them, I think that it helps.

Rebecca: So do you want to share some of the results that you got from your study?

Sarah: We saw some very significant quantitative results. I mentioned the MAPS instrument is what we use. It’s a 31-point scale. Its reliability and validity has been established pretty well, especially in calculus classes. One of the things that they did was they looked to see if the items were consistent with expert consensus…. So, with how mathematicians view it and all of the items were valid with the attitudes of mathematicians except some of the growth mindset scales. Research says that that’s an important scale as well. And on this 31-point scale, we saw an almost 4-point improvement from pre-test to post-test…of the students becoming more aligned with the expert opinions, which is a really significant amount…I mean, almost 10% improvement, which is even more remarkable, because when this assessment was first validated, they found that there was usually a negative result from taking a Calculus I class. So, the attitudes get worse pre-post in a calculus class and ours had statistically significant improvement. In addition, we saw statistically significant improvement among all of the sub scales. Now some of them were better than others. Some were just barely below .05 in terms of significance and others were much more significant. I mean, we really saw that over the course of this semester, they really did change their attitudes. We also had some evidence, as John’s already talked about, from their essays…where they said how they started to view mistakes as productive, and they started to feel like there was value in making mistakes and learning from them.

John K.: You mentioned alignment with an expert scale, can you explain that for our listeners?

Sarah: Essentially, what the original authors and it was Code et. al. that did this paper and develop this instrument. They gave this survey to students and they gave it to mathematicians and looked for alignment. Particularly they were looking for whether or not the mathematicians agreed on the items. And the idea was our goal is to get math students to have attitudes more like mathematicians, because that’s our goal, right? …is to develop future mathematicians. And so we would like those attitudes to get closer to how mathematicians view mathematics. They had high agreement among the mathematicians on every item, like I said, except one or two of the growth mindset questions. So, in other words, this survey reflects how mathematicians view mathematics. And that was how they determined the right answers on the survey, whether a particular item is something you should agree with or something you should disagree with. They went with the expert consensus.

John K.: So now, I may be misconstruing this, but are you suggesting that perhaps a lot of mathematicians had adopted a fixed mindset? So, there was a bit more variance there on that?

Sarah: I will say that was what the results of their validation showed.

John K.: Okay.

Sarah: And leave it at that. [LAUGHTER]

John K.: It does remind me of that study a few months ago, that found that when instructors had a growth mindset, the achievement gap narrowed and the drop-fail-withdrawal rate was much lower in courses, then for those instructors who had a fixed mindset. I think that maybe even more of an issue in the STEM fields than it is in humanities and social sciences, but I think it’s not uncommon everywhere.

Rebecca: I say it’s a common problem everywhere.

John M.: I’ll say it…mathematicians suffer from fixed mindsets. I’ll just say it, right? [LAUGHTER]

John K.: Many academics do.

Sarah: Yeah.

John M.: Yes, of course.

Sarah: I mean, the people who choose to become academics are often the people that were successful in school and they decide to continue with it. I mean, it is less likely that people who felt unsuccessful decide to keep going and to go into academia.

John K.: Selectivity bias there and that reinforces a belief in a fixed mindset, perhaps.

Sarah: Precisely.

Rebecca: What kind of response have you seen from students from…I mean, it sounds to me like this one study lead to good results, and then that changed many classes in that you’ve taught or the way that you’re teaching, how have students responded?

Sarah: Generally positively. I think doing the projects at the end of the semester wasn’t the best idea because they just feel so overwhelmed at the end of the semester with exams and projects and everything coming due. So, I did get some responses of “W hy do I have to do this now.” But generally, I think they appreciated learning about learning.

John M.: I think that given the opportunity to talk about their past experiences, I think they appreciated that. For the most part, I’ll agree with Sarah. I think that the message landed with an awful lot of students like I wanted it to. Some of my favorite essays were students who told me that they thought I was crazy on the first day. I mean, you go into a math class to learn math, you don’t go into a math class to study metacognition, or whatever it may be. I had one student the first time around, who basically told me it was all a load of crap, like why this is not working at all. And I had a student the last time that I did this, she was very skeptical towards the end even. Basically, aliken it to just some cheesy self-help stuff. I think that most students responded positively.

Rebecca: Have you seen the response impact other faculty in your area? For example, if they really liked having those techniques and things introduced in your class, have they asked other math faculty to do that in future classes or are you finding that its not many math students who were actually in that particular class?

Sarah: We haven’t done any tracking, so I don’t know where his students have gone. I mean, I’m sure some of them went on to Calc II…I’m sure some of them did not. Right. I mean, I guess most of them would have had Jess the following semester, right? Did she say anything?

John M.: No, she didn’t say anything. I’m teaching Calc III right now, and I have some of my former calculus students that were in this and they’re doing well.[LAUGHTER] Small sample size, but yeah, they’re doing well.

John K.: That could be an interesting follow up though to see how successful they were in the subsequent classes.

Sarah: Yeah.

Rebecca: Sometimes we’ve heard anecdotes, of departments and things when there’s been change that if students really respond well to whatever the techniques are, that they will demand it of other faculty members, and John’s talked about this before in economics.

John K.: Yeah, when you can show results…

Rebecca: Yeah.

John K.: …that there’s been some gain, and especially if it comes from students at the same time, it often puts pressure on other people in the department because if you’re able to show people that your technique has been successful and students are coming in and saying, “G ee, I wish you would consider doing this. I did this in my intro classes, and it was really helpful.” That sometimes helps make change much easier.

Sarah: Yeah, so one of the things that we did look at was we compared the final exam scores of John’s sections to the other sections of calculus that semester. Now, there was some other issues that clouded that data a little bit. His scores were a little bit lower than the other instructors. But what was really surprising, essentially, if you look at, I don’t remember if it were just the final exams or the semester grades. The DF rates were the same among the sections, but the withdrawal rates were significantly different. And that almost no one withdrew from John’s sections. I think there were two if I remember the data correctly, whereas there was like five or six on average from the other sections. And so the DFW rates were different, but the DF rates weren’t. So I just thought that was an unusual circumstance. So, it seems like the students were sticking with his class… and pushing through.

John K.: And if there is a larger portion of students staying with the class, then perhaps a slightly lower average grade is not necessarily a bad sign…

Sarah: Exactly.

John K.: …because student success is partly measured for persistence to completing the course.

Sarah: Exactly. I think because there were more students who stuck it through to the final exam, then his final exam scores ended up being a little bit lower. But again, if you looked at like overall course grades, they ended up being pretty consistent, other than the W rates. I wanted to make sure that there weren’t significant differences in the rates and I think it was just shy of being statistically significant. Like, if you had one more student that would’ve been significant. But just to make sure that, especially like adding the test corrections in wasn’t substantially making the class too easy, right? Because that’s often a critique that, you know, “Well you make these changes, but is that just making the class too easy and people who aren’t really prepared, are they passing?” And so I just did this analysis of the, like I said, it was really just a t-test analysis, but just to see whether or not it was significantly lower and it wasn’t significant. It was lower, right, just not significantly. And then like I said, I looked at retention rates just more as an explanation for why the average was lower.

John K.: In a lot of studies of interventions, the dependent variable is the drop-fail-withdrawal rates, because that’s a measure of success in completing the course. That by itself could be an interesting focus of a study. I’ve been running this metacognitive cafe in my online classes for a while and I did have a student in the class who wrote a few times about the metacognitive development that was introduced in one of your classes. They didn’t specify who but they said, we’re also doing some work on metacognition in the math class, and they said it was really useful and it was nice to see it in two classes.

Sarah: Yay!!

John M.: Good.

John K.: So there’s at least one positive data point there or one additional data point there. So are you going to continue this in the future? And if so, what might you do differently?

Sarah: Well, I think we’ve mentioned already that we’ve worked on including some of the ideas at the beginning of the semester and throughout the semester, rather than one project at the end. For the reason that it really benefits them most at the beginning of the semester when things are getting started. I think we’ve also both changed different things about our grading systems to incorporate more opportunities for growth.

John M.: The last time I did this, I introduced some articles that were a little bit more rigorous with the data and the science, because I sort of wanted to counter that kind of criticism that all this “Oh this is just a bunch of TED Talks…” that kind of thing. So, I really wanted the students to see some of the science behind it, the science of learning, because I really wanted to send that message that “No, this is not me just standing up here saying, ‘Oh, this is going to help you or anything, right?’ This is actually stuff that researchers have thought about before.”

John K.: I had a very similar response the first time I did this. I had a video I posted which was a TED talk by a cognitive scientist who talked about research that showed that learning styles were a myth. And some students had come to believe in the existence of learning styles because they’ve heard of them and often been tested, multiple times in multiple years, on their learning styles. Sometimes even through college and that’s rather troubling. The students said, “Well, this is just one researcher, I’m sure there’s lots of other studies. I don’t believe it because it’s not consistent with what I’ve always been told or what I’ve heard.” So I decided to modify it then and I added to that discussion, five or six research studies. In case you don’t believe this TED talk by someone who’s done a lot of research on this, here’s a number of studies, including some meta analyses of several hundred studies of this issue, and that has cut much of that discussion. They’re less likely to argue against it when it’s not just a talking head or not just a video when they can actually see a study even if they don’t understand all the aspects of it.

Sarah: Yeah. So I think that’s one thing we’ve tweaked what articles and what videos are we showing. I know the semester I gave my students a article that had just come out this September, that students perceive active learning as being less efficient, even when they’re learning more. In some physics classes at Harvard, they gave two weeks at each thing… two weeks of active and two weeks of lecture, and then they had them switch. And the students learned more with the active learning, but felt they learned less. And my students have been feeling frustrated because they feel like they’re not learning enough and that I’m not telling them what to do.

Rebecca: You’re not “teaching” them.

Sarah: I’m not teaching them. And we spend the class period, letting them vent. So all their feelings were out in the open. But, then I sort of countered with this article saying, “Look, I promise you really are learning things. You just don’t feel like you are. But you really, really are. And you’re actually learning it better than if I were using a different style.” So, that’s one way that we’re tweaking the articles because sometimes the research comes out that’s pertinent.

John K.: We refer to that Harvard study in a few past podcasts. We touched on it in a podcast that will release on October 9th. I haven’t shared it with my class yet, but I’ve been tempted to.

Rebecca: What was the discussion like talking about that particular article? Given that they were frustrated?

Sarah: I mostly was just trying to acknowledge that I understand their frustrations…and that, yes, the way I’m teaching this class can be frustrating. I agree. Sometimes I get frustrated about it. But I know that ultimately, they are learning things and that they are going to be stronger writers and stronger students of mathematics by using this structure. And so I kind of use it as evidence of I’m not changing.

Rebecca: So I hear you…

Sarah: Yeah.

Rebecca: …nut…

Sarah: I hear you, but…

John K.: I had this very conversation with my class today. They’re coming up for an exam very shortly. And I asked them, how did they review before an exam and the most common answer was they like to reread the material over and over again. And I mentioned some of the research on that. And I said, the best way to review is to work on problems with this. And I gave them several ways in which they could do that, that are built into the course structure. And I said, “But that doesn’t feel as effective. Why?” And one of the students said, “Well, I get things wrong.” And I said, “And when would you rather get things wrong, when you’re reviewing for an exam, or when you’re taking exams?” And I think some of them got that message. So I’m hoping we’ll see when they take the test next week.

John M.: Right? It seems like anytime you do anything that’s just not a standard straight lecture, there’s a certain amount of buy in that you need to get from the students. And sometimes that can be very difficult. There’s almost a salesmanship that you have to do throughout the semester to make sure that everybody’s on the same page and to kind of fight those feelings where the students give you a lot of pushback. Yeah, that’s the great fear is that when you innovate or you experiment that’s going to go horribly wrong. And sometimes it does, but, you know, we still keep going.

John K.: Because students are creatures of habit. They’ve learned certain things and they want to keep doing things the same way. And anything new can seem troubling, especially if they’re getting feedback along the way that says they need to work more on things…that’s not as pleasant as rereading things and having everything look familiar.

John M.: Right

Rebecca: Passively sitting in a lecture when things all seem like it makes perfect sense to you, because an expert is describing it who knows what they’re talking about, right? Always feels easier than trying to apply it yourself. And I think that students, even though the lecture might feel better, and learning is hard…over time…at the end, when they’ve seen how much they’ve accomplished, and you do have them reflect…many of them appreciate or come around. Sometimes, it’s not in that same semester, sometimes it’s emails, months or years later.

John K.: Yes.

John M.: Right. Right, right.

Sarah: If only if we could do course evals, you know, a whole year later,

John K.: Or five years later. That may not work too well in my tenure process, though.

Rebecca: We always wrap up asking what’s next?

Sarah: Well, the first thing is we’re hoping our article gets published. It’s been submitted. We’re waiting for reviewers. I’m going on maternity leave next semester…that’s really what’s next.

Rebecca: Sounds like a new adventure.

Sarah: It is a brand new adventure.

John M.: Wow, I don’t think that far ahead, I guess. Yeah, I guess I’m that unoriginal, huh. But, yeah, no I’m just trying to…

Sarah: We’re moving to a new building.

John M.: Yeah, moving to a new building, and getting a new department chair. Yeah, that’s right.

John K.: A new desk to go with the chair?

John M.: No. Ah… Yeah, funny, funny, funny.

Sarah: if only…

Rebecca: Well, thanks so much for joining us, this has been really interesting.

[MUSIC]

John K.: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John K.: Editing assistance provided by Brittany Jones and Kiara Montero.