358. Essentials of the New Science of Learning

Most freshman students enter college with little knowledge of evidence-based strategies for successfully navigating the college experience. In this episode, Todd Zakrajsek joins us to discuss a variety of approaches that students can use to more efficiently achieve their learning goals. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years. His most recent book is Essentials of the New Science of Learning.

Show Notes

Transcript

John: Most freshman students enter college with little knowledge of evidence-based strategies for successfully navigating the college experience. In this episode, we explore a variety of approaches that students can use to more efficiently achieve their learning goals.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Todd Zakrajsek. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published six books (so far) in the past five years. His most recent book is Essentials of the New Science of Learning. Welcome back, Todd.

Todd: Well, thank you. It’s great to be back.

Rebecca: Today’s teas are: Todd, what do you got today?

Todd: Oh, today I have a white ginger pear.

Rebecca: Nice.

John: Very nice. And I am drinking something similar, a ginger peach green tea.

Rebecca: Nice. I’ve got Cardamon Cinnamon.

John: Nice.

John: We were just talking to you a few months ago about your last most recent book, but we’ve invited you here today to discuss your newest most recent book of the year. Can you tell us a little bit about the origin story of The Essentials of the New Science of Learning: The Power of Learning in Harmony with Your Brain.

Todd: When I worked on The New Science of Learning: Learning in Harmony with Your Brain, that book, its third edition has gotten up to be about 265 pages, and the book was actually written for students. It is still for students, but when it started, the whole concept of this book is it would be used as kind of an ancillary text in a class. And a friend of mine out in California used it in a community college in a developmental math class, and they found, actually, when they used the book in the class, that the attrition rate went down. And for developmental math, that’s a very important concept. In fact, the students who did pass for the next semester, the students who had learned about the science of learning, that attrition rate was even further down. So it’s helped out a lot. The problem is, once it gets up to 265 pages, you can’t use it as kind of an ancillary book. You know, “Read this chapter, it’ll take you 10 minutes.” The New Science of Learning is built more to be like a primary book for a first-year seminar course. So this book, being The Essentials, allowed me to come back and add some new things about AI and a few other things. Is kind of a streamlined version of that that can be used in a class, so that you could say to your students, for instance, “Hey, on Wednesday, make sure you read the chapter on sleep,” and then you talk about the content. When Wednesday comes along, you spend five minutes talking about sleep, what did you learn? And then you move on with the content. And that’s how you can infuse it in the class.

Rebecca: Why isn’t this material taught to students earlier in their education process.

Todd: Aaaah… this is a question that we’ve talked about before with other kinds of science of learning things. It’s frustrating. I take the breath because it’s frustrating. It’s funny, across country, if you ask students two questions: “do you like to learn?” and “do you like school?” you tend to find that the students will say yes to both those questions until they hit right about third grade. In third grade, we start picking up some of the testing and really focusing more on content, and at that point, we start telling students, you need to learn these things, but we don’t tell them how. There just is this assumption that because people are learners, that they can learn it. But learning out of a book is very artificial in terms of how we evolved. So the concept is students should be taught that, and I really believe they should be taught that starting in about third grade. And could you imagine a student that comes into college that has spent eight or nine years learning how to learn along with the content that they have? I think it would be amazing, but typically not done, so it should really be done as soon as they get to college and as quickly as possible. And it doesn’t matter who I talk to. I have talked to students from all kinds of disciplines. I’ve talked to students in medical schools, nursing programs, a lot of the health professions, they do a ton of content really fast. I’ve talked to students in philosophy and history and psychology, and almost invariably, people will come up afterwards and say, “Why doesn’t anybody teach us this stuff?” And so the students see value in it too. We get focused on content, and we don’t tend to do it.

Rebecca: It’s interesting, Todd, that you’re saying that there’s a wide range of people that are hungry for this information because before we were recording, I was doing graduate student orientation. I was talking to a couple of MBA students who had been out in the field, into their career 10 years, coming back to school, and I had embedded some of this kind of information in their online orientation. They said, “Oh yeah, I watched this TED talk that you had included [LAUGHTER] about the science of learning, and it was really great. It was so interesting.” And I had included that study from Michelle Miller.

Todd: Oh, yeah.

Rebecca: And he had read it, and it was fascinating. He’s like, “I was so surprised that I didn’t know some of this stuff.” So we’re talking about it at lunch today, and they were all talking about, “Yeah, that was really interesting. I’m so glad you included that.” But it just goes to show that it doesn’t really matter who our learners are. They’re really hungry for this information, and it’s really valuable to them.

Todd: Yeah, I will say quickly, in terms of the book that a lot of this is drawn from, which is The New Science of Learning, the full version of it that I have at the end is, if you find this valuable, or if you’d like to just ask other questions, I put my email address in there and just send me an email. And I got an email like, three weeks ago from a student who’s going into a health professions field, and he’s going in as a post grad. I mean, he’s going in as a master’s student, and they’re all required to read that book coming into the class, so it’s at any level. In fact, faculty find a lot of the stuff new and interesting to them, so we should teach it to everybody. Somebody should write a book about it.

Rebecca: Huh.

John: And fortunately, you have.

Todd: Hey, [LAUGHTER] me and a couple other people have too.[LAUGHTER]

John: So you mostly focus on effective learning strategies and how students can use the time more efficiently in learning. You also provide a lot of other advice to students about challenges that might arise for different students or students in different categories. Could you talk a little bit about some of the other advice you provide in addition to those directly related to learning?

Todd: Yeah, there’s a lot of other things that I think are really important. There’s a chapter on how we learn and a chapter on how we remember things, which learning and memory are separate, and those talk about the structures and the way we do that, but there’s so many other things that come into play, and a lot of things from social psychology. Everybody talks about cognitive neuroscience and cognitive psychology, but if a person understands exactly how to learn. If you’re really, really frustrated at the time you’re trying to learn, it doesn’t matter the actual cognitive structures, because the social part of it just wipes that out. And so things like microaggressions, how we perceive other people and how we interact with other people, has a huge impact on learning. Stereotype threat, whenever we get in a situation, if we’ve got a person who is a minority within a class. So it could be a male in a nursing program with mostly women. It could be in some discipline that has mostly men with a woman. It could be a student from an underrepresented group. When there’s an individual there that doesn’t typically come into the play, they’re under this pressure all the time that if they do poorly, then it reflects on “Well, maybe men can’t do nursing,” and so there’s extra pressure on them, and they tend to struggle with that. And things like imposter syndrome, almost everybody I’ve talked to at some point or another says that I don’t feel like I really belong, or I don’t feel like I know as much as other people think I know. And so those types of things, along with goal setting and dealing with or helping people who are working. I mean, what happens when you’re trying to work? Huge proportion of our students now work part time, but there’s a large portion also that work full time in addition to being full-time students. So a lot of those things can be drawn into this whole concept, and all those things are in the book.

Rebecca: One of the topics that you discuss is managing cognitive load. Can you talk about some of the recommendations you make related to this?

Todd: Yeah, cognitive load is one of the foundational things. And if you’re a listener and do not understand what cognitive load is, one of the best things you can do is get up to speed on that a little bit. A place to start, I always say, is Wikipedia. And if there was a motto that I would throw in there, it would be Wikipedia, great place to start, and a hideous place to finish. And I just want to point that out there, because there’s some stuff you can get pretty quickly. But here’s the thing with cognitive load, is it basically boils down to there’s only so much information your brain can process at any given time. And if you try to do too much, then it pretty much shuts down enough in one area so that it can process a different area. Be a little bit like if you’re putting together an IKEA bookshelf for the very first time, and your significant other comes into the room and says, “Hey, where do you want to have lunch?” And you may very well say, “Not right now,” and your brain’s just full with all of these parts and how things are supposed to fit together. Once you put 10, 15 IKEA bookcases together, you could carry on a conversation with somebody else while you’re doing it. And that’s an all cognitive load issue. The first time you do something, it is taxing, and it takes extra energy for your brain to sort out all the pieces. But the more you do something, the easier it becomes. And this is why, as an expert, you have to be careful. Because as an expert, I might say to my students, read this article, this 10-page article, and I might read that 10-page article very, very quickly, because I’ve read this kind of stuff over and over and over again, but the novice is going to take way more time because it’s their first time doing it. So we can talk to students about the fact that this is going to be hard when you start, and then as you do it, more and more, it’ll be easier, and that’s a cognitive load issue. As you actually work on things as combinations, as things come together, you build what are called schemas, and those are like scripts that all come together, and as you activate a whole schema, then that’s just there. If I go golfing, I don’t have to think about every little piece of that. Or if I’m driving, I don’t have to think about how hard I turn the wheel and turning on a turn signal and everything else. I have a schema for being in a car and the types of things you do. So as we practice and go through these things, that’s the cognitive load issue. There’s going to be heavy cognitive load, which we have to be cognizant of, and then, as we do it, it becomes easier. And then there’s another piece to this. Is if there’s things in your environment that don’t help you to learn, but they take up cognitive energy, so a conversation by people on the other side of the room, pretty much everybody in here listening this has probably, at some point been reading something, and people are talking, and you either say, “Excuse me, could you quit talking?” Or you pack up and go someplace else. If you’re reading a hard article, that will happen. If you’re reading something like a novel, people can be all over the room talking, and you don’t care. That’s a cognitive load issue. Hard article, high cognitive load, an easy thing, lower cognitive load. This is one I will jump in there and say, too, this is an area that AI has me very concerned. Because as you start anything, you start at the foundational level, and it’s challenging, but as you work with it over and over again, it becomes less and less challenging, then you can move up to more complex things. If AI is doing the simple foundational stuff for us, then we don’t develop that base that allows us to kind of automatically process something, so that we can move to a higher, more complicated thing and then practice that and go to a higher, more complicated thing. If AI is doing a chunk of it, we don’t ever develop the base of it. And so I’m concerned with respect to cognitive load development and how AI might come into play with that, but that’s the two-minute version of a topic that we spend two or three class periods on [LAUGHTER] in a course in psychology.

John: One of the many other topics you address in this book is metacognition. Why is developing metacognition so important? And what are some strategies that students can use to increase their metacognition?

Todd: Yeah, cognitive load is huge. Metacognition is right in there next to it. And metacognition basically boils down to knowing how much you know, knowing how much you’re thinking about something. It’s the process of thinking about the process, and we often don’t do that. It’s actually kind of fascinating when you chat with people, is they’ll engage in a behavior and not think anything about what they’re doing. So we fall into patterns. And so I’ve had students for years that will come in and say, “You know, Dr Z, I’ve read the chapter like four times and I’m still struggling.” And I’ll say, “Well, why did you read it four times?” It’s like, “Well, I need to to learn it.” “Isn’t there any other way you could learn it?” And as I’m asking these questions, I’m trying to bring up a metacognitive concept of let’s reflect on your learning process. If you’re struggling with the material when you read it, it may very well be that two, or three, four more times… In fact, research has shown that you’re probably not going to learn anymore, because if you’re not paying attention to your cognitive functioning, you may very well be reading while you’re daydreaming about something else. And the reason I mention that is that that is not being aware of your processing of information. So again, something that all of us have done, you finish reading a chapter of a book, or you finish reading an article, and just as you get to the end of it, it’s like, “Oh, my word,the entire time I was thinking about bacon or lunch of some kind.” And if that’s the case, you’ve just wasted that time. So for metacognition, it’s kind of thinking about how you’re studying, evaluating it, to what extent are you successful When you sit down and study for an hour, and if you quiz yourself, do you know more or don’t you? So practicing with that, thinking about while you’re studying, what’s going well and what’s not going well. If you’re frustrated in your surroundings and you’re having trouble processing, then you can make changes. If you don’t think about it, you can’t and then also thinking back. So after I take a test or after I study, I can think what went well and what didn’t go well. And that’s all kind of metacognitive things that, because of the way we live, we just don’t tend to do them. And so you know people who will repeat the same pattern over and over and over again, and we use phrases like, “You’ve done this four times, it’s always failed. What were you thinking?” And when we say “What were you thinking?” we’re actually asking a person, to what extent are you engaging in metacognition. But quite frankly, the “What were you thinking?” it’s such an automatic statement that it really doesn’t pertain to thinking.

Rebecca: Feel like a next interesting leap might be to talk about the myth of learning styles, which is something that people often cycle through and say often as well, like, “I’m a visual learner, and this isn’t in a visual style, so I can’t possibly learn it.” And this is something that you address in your book as well.

Todd: Yeah, I think foundationally, first of all, for anybody listening, I’m pretty sure you probably know about learning styles, but if you don’t, there’s a whole concept out there by Neil Fleming came up with this in terms of the learning styles index. And essentially, you take 13,14 quiz questions and you answer these questions, and based on how you answer them, you’re categorized as typically, either a visual learner, an auditory learner, or a kinesthetic learner. A couple of foundational things. Number one, anytime there’s some kind of a test that you can take on Facebook or online or in a quiz book or someplace or even somebody reading 13 items to you, and that’s supposed to tell you how you function as a human being. We’re way more complex than that. Psychologists will ask hundreds of questions to come up with a response that says based on the hundreds of questions that we have, all these validated questions, we believe you will tend to do this. Now, think about that for a second. That’s after a whole battery of tests. They’re saying we think you tend to do this, which has got a lot of hedging in it, which is different than 13 items, “Oh, you’re a visual learner.” So keep in mind that it doesn’t work that way. Number two is that there’s nobody who scores like every item is all on visual or every item is on kinesthetic. If you tend to answer, let’s just say 40% of your answers are on visual learning, and 30% are on auditory learning and 30% are on kinesthetic learning, you’d get categorized as a visual learner because that had the highest percentage. But really it’s a 40-30-30. Why wouldn’t you be categorized as all three? But boxes are easier to deal with, so the learning styles has lots of concepts behind it. Research will also indicate or show you pretty quickly if you read through this stuff, it’s just not how learning works. I would just love to say this, because I don’t want to just say learning styles isn’t good. Here’s the concept Is basically the best way to learn is to practice with the material in a way in which you want to retrieve it. If you want to actually dance, the best way to learn is kinesthetically, by dancing. You might get some auditory or visual information of here’s what the pose looks like, but by and large, you’re going to want to dance. If you’re going to want to be able to recognize certain kind of prints from Monet versus Manet or somebody else in painting, then probably the best thing to do is look at a lot of paintings. Visual would make sense there. Semantic learning, if you want to remember something, you’re probably going to want to read it or hear about it, and so you want to practice it in the way you’re going to retrieve it. The reason people love this concept of learning styles is people love to take quizzes that put them into boxes. I don’t get this, but people take quizzes all the time. Here’s 10 ways to determine what kind of a spouse you are, or five things you can do to figure out what kind of parent you are. We love to do those. And the problem is that learning styles comes along and people think that’s how I learn. And based on what you said at the beginning, Rebecca, when a student says, “You don’t use many images and I’m a visual learner, therefore I can’t learn from you,” they won’t learn from you. But they won’t learn from you because they’re a visual learner, they won’t learn from you because in their head they’re thinking, I can’t learn from you. And when your brain is processing, “I can’t learn this. I can’t learn this. I can’t learn this.” there is so much cognitive load dealing with the fact you can’t learn it there’s no room for learning. So if you think of yourself as a visual learner, I would say, just think about it in terms of you like to use visual things to help you to learn, but there’s a lot of other ways that you can learn as well.

John: Going back to an earlier location in your book, one of the things you talk about as a suggestion for students is that they avoid dichotomous thinking, and you also add in a discussion in the essentials book of generative AI. So this is kind of a mashup of those two concepts. But will the use of generative AI be helpful or harmful for student learning?

Todd: Yes, [LAUGHTER] yeah, the dichotomous thinking, I will say, just to address that very quickly, is that whole concept is that it’s the weakest kind of thinking. It’s basically you put somebody in a box and you don’t have to deal with them anymore, because this is them. That person’s a very aggressive individual. Everything gets interpreted as aggressive because that’s who they are. We don’t stop and think about on a given day, maybe a lot of bad things have happened; on a different day, maybe good things happen. So maybe they tend to be more aggressive, but they’re not just yes/no, you are/you’re not. So AI… people have asked multiple times, is this terrible or is this good? I tell you, everything has nuances. So the bad side, I do think AI is going to be really tricky for us, because humans, in fact, all organisms, will take the easiest route we can. In many respects, we should. If we are going to work and it’s five miles away and there’s a car in the driveway, taking the car makes a lot of sense. So I should take the easy way. However, there’s going to be times when I’m going to want to be physiologically in good shape. I might need to run. I might want to walk up a flight of stairs. I might just want to live for a while, in which case I should work on helping my heart to be better. And the human body is fantastic, and the more you push something, the more it will develop. And so I love to ask students, for instance, I’ll say, “Anybody in here run or jog?” And a whole bunch of hands will go up, and I’ll say, “Why? You got cars. You should just take a car.” And somebody will say, “Well, yeah, but I want to be healthier.” I’ll say, “Perfect. Let’s talk AI for a minute.” For AI, if we let the AI do all the things for us, then we don’t develop those things. And so if I’m taking a poetry class and I just keep having AI write the poems, write the poems, write the poems. It can do it. I can edit them, but I’m not going to learn how to write and how to express myself. And so the issue we’re going to have is, if we don’t learn how to do those things, especially at the beginning level, kind of the cognitive load thing, we’re not going to be able to do them when it’s more complicated. So I think we have to be very careful of that. And for faculty, I think we have to be very careful about focusing on things like academic misconduct. Yes, there are students who will use AI to cheat, but students didn’t start cheating because AI came along. Based on the culture in your classroom, the rapport you have with your students will make a huge difference in the extent to which students will engage in academically honest behaviors. So anyway, those types of things, we’ll watch out for. The good side, there are so many things that this is going to be able to use for good. For instance, we know from study after study after study that the more you practice at something, the better you become. And we have said, I have said for years too, the best way to learn anything is to teach it to somebody else, if you don’t have a family member or a roommate, that you can teach it to, then teach it to a dog, teach it to a sofa, just teach it. And the concept there is, the more you explain something, the better you get at it. AI is beautiful for this, because it can work with you. Actually, I tried something recently I thought was so much fun. I told AI, and the scenario in my head was, I’m a student in a psychology class, and I just learned the routes of persuasion by Cialdini, and now I have to remember how this persuasion all works. Well, one of the best ways to do that is to practice with it. And so I fired up ChatGPT and the prompt I gave it was, “Hey, how about if we do a role play? I’m going to be a car salesman, and you’re going to be in the market to buy a car. You do have enough money, you could buy today, but you want to be a little cautious, because you want to get input from your family, and you want to make sure you get a good value and get the kind of car you want. So overall, if I really can sell you, you’ll buy, but otherwise you’re going to wait. What do you think?” And then ChatGPT typed out, “This sounds like fun. Can you show me some vans?” And so then I said, “This is a van,” and I started to sell it. Now, the reason I’m mentioning this is that I could use all my different persuasion techniques that I learned, and ChatGPT kept asking me questions back. So I’d say this van has something about scarcity. Here’s a good one for scarcity. It’s one of the principles to persuade people. I said, “Whis is the last van on the lot. And so ChatGPT said, “Oh, that’s unfortunate, because I’m really interested in the van. Is there a possibility of getting another one if this one sells before I actually sign the paper?” And now what ChatGPT has just done is to see if they could reduce that concept of basically a quantity issue of scarcity. If this is the only van, a human would basically say, “Oh no, it’s the only van. I better get it before somebody else does,” so with ChaGPT, it says, “Can you get another one from a different lot?” And if I said, “Yep”, then the scarcity goes down a little bit. So then I have to think, “Oh, how can I bring scarcity back in a different way? It’s on sale, but the sale is going to end at the end of this week, so we’re going to have to make a decision fast… that scarcity in a sense of we don’t have much time. And so the point is that I role played with ChatGPT for about 20 minutes on these principles of persuasion. We’ll be able to do that. We’ll be able to do simulations. We’ll be able to do repetition. I can say to ChatGPT, and I’ve done this one. “Give me 10 multiple choice questions about metacognition for a student in a second-year college course, and the course is on the basics of cognition.” The better prompts you lay out there, the better it will do. It asked me multiple choice questions. And the last one I’ll mention for this, because there’s so many different ways you could do it, I actually gave it a definition of metacognition. I wrote out kind of what it is, and then I just put it in ChatGPT, and I said, if I answered a question of “What is metacognition?” with this response, and I was a senior in a cognitive psych course, because it’s different from being a first-year student, but it’s going to give you feedback: “If I’m a senior in a cognitive psych course, what grade would I get?” And I purposely made it bad, and it said you would get an “F” on this. And then I said, “Why?” And it explained a couple reasons why. And then I rewrote it, one of the best responses I could possibly dream up. And then ChatGPT said, “this is a good solid A minus.” And I said, “Why is this an A minus?” And it pointed out that I hadn’t done very much on the reflective component. And I thought, “hmm, alright, I did miss that piece.” And so when I can practice quiz questions like that, I can have it ask me multiple choice questions. And much like Khan Academy, when Sal Khan came up with Khan Academy, he said, one of the things that students love about this is you can go and watch the same video four or five times, and it doesn’t get frustrated with you. ChatGPT, you can struggle with learning a concept, it’s never going to get frustrated with you because you’re struggling. Whereas a human might say, in their head, it’s like, “Oh my word. You should have gotten this by now.” So I know that was a very long answer. It’s a huge, huge issue in education right now, but we can’t be focused on the academic misconduct side of it. We really need to be focusing on how can we use it to instill deeper learning?

Rebecca: That’s a nice transition to one of the other topics that you have in your book, which is maybe something that faculty might be interested in knowing is in your book, [LAUGHTER] which is that you advise students to spend two hours outside of class for every hour they spent in class, which is definitely an expectation, I think, that many faculty have, but it’s not always conveyed to students. I know I mention it to students, [LAUGHTER] but can you talk a little bit about that?

Todd: Yeah, so I put it in there, and it was funny because it was a section I put in the book, and one of the reviewers looked at it and said, “Why did you put this in here? And then it was funny because the editor was reading the book and said, “I love the fact that you put the breakdown in there about how much you should study and how much time you should commit, because people don’t normally know this stuff.” And so I put it in there because I think people struggle with this, and for students to understand, if it’s a course that requires extra work and things, if it’s a three-credit hour course, you’re expected to do two hours outside for every hour inside. This is the way the system was built up. So a three-credit hour class is going to take you about nine hours a week. The three hours in class, then the two-to-one ratio will give you six hours outside of class. This is why, if you take 12 to 13 credit hours, or 13 to 14 credit hours, it’s considered full time, because 13 credit hours would put you up around 39 hours total, when you put all the hours together, and that’s a full-time deal. And by the way, just so I don’t forget this, or I put it off to the side, if it’s an activity course, something you wouldn’t do anything outside, that’s why a one-credit hour class might meet for three hours, like a jogging class or something, racquetball class, because there’s not outside work. The two-to-one ratio kind of thing, all of that’s done in class. So that kind of gives you a sense of how much you should be studying. Here’s the tricky spot, nobody does it. I shouldn’t say nobody, very few people do it. The reason is that a lot of students can get by with studying less than two hours outside for every hour inside class. And when students will say they don’t need to, or faculty will say they don’t do it, and people focus on the fact that, “Well, you know, students don’t really have to spend that much time.” I love to respond with, oh, so they all are getting 100% on their quizzes and tests and perfect papers, right? And to that, we say, “Well, not really.” System’s set up that you expect a lot of B’s and C’s and D’s. Well, what if you spent two hours outside for every hour inside? It would be a very different thing. But the problem again, is, “Whew, if students are not spending the two hours outside for every hour inside, and then you are, then it seemed like you’re taking more time, and you feel like, why am I spending all this time when they’re not?” And so we look to other people, and unfortunately, we don’t do that. But I can tell you, for students, if you do actually sit down and study for two hours outside of class, for every hour inside class, and do it before the class period. When you walk in, you already know material there. You understand the foundation of it. And if the instructor is doing a good job, they’re pushing you and showing you connections and everything, you can process that because you have the material. If you walk into class after skimming a Chapter, you don’t have the foundational material to make the classroom more effective for a learning situation.

John: I was really happy to see that advice there, because I tell students the same thing, and they always seem really, really surprised. Or in an online class if you tell them they should expect to spend nine hours there, there’s often an even larger surprise, because they think it’ll take as much time as what they spend in high school, which is often zero outside of class and with an online class, zero doesn’t work very well, especially. So I’m glad you included that. But one of the other things you include in your book in terms of advice to students is you talk about various fallacies, and one of those is a sunk cost fallacy, which is something that we talk about quite a bit in economics, especially behavioral economists. So could you talk a little bit about this fallacy and how it might affect students?

Todd: Yeah, sunk cost is something that I always covered in psychology, because I love the concept, psychological concept. It’s really important a sunk cost basically just simply means, how much energy or resources have you put into a course of action. That’s basically it. How much have you sunk into this thing? The problem is for humans, is we overestimate the value of that. If you’ve worked on a paper for six hours and it turns out you’re heading in the wrong direction, best thing you can do is pitch the paper and just start over. But people say, “Well, wait a minute, I’ve put six hours in on this. I must be able to salvage something of it.” And I watched a woman on television years ago, and I think it was Ukrainian eggs. It was when they hollow out the eggs, you get the insides out, and you cut half of it away, so you’re dealing with, like a little half of a shell. And they do these beautiful, intricate things inside the shell. And she pointed out, she said, “If you ever drop one of those, the best thing you can do is stomp on it,” because if you drop it and it cracks, you try to fix it, and you spend so much time trying to fix it, because the sunk cost is, “I’ve already put hours into this,” then you waste a lot of time trying to fix it, your better bet is just to throw it away and start over.” And that’s why she said stomp on it, so you’re not tempted to do that. So in your work, we’ll do this all the time. We try things, and we get all this investment… we might be in a class that’s just we’re flunking the class, and this is always hard, and you always want to always talk to an advisor if any students are listening, or tell your students always talk to an advisor before you ever drop a class, because it can have ramifications, especially if it drops you down to part-time status. But if you are going to flunk class, the best thing you can do is just stop working in that class. Take the F if you’re past the drop period, and just focus your energy somewhere else. And students will often say, “but yeah, but I’ve already put so much time in this.” And when I was teaching intro psychology, this is a particular one is students would come in with like, 50% average and come into the final, and I’d say, it doesn’t matter, you’d need a 98% to get a passing grade. You’re going to get an F. And the students say, “Well, yeah, but I put so much energy this I’m going to try” and I’ll say “If you’ve averaged 50, you’re not going to score above an A plus. It’s just not going to work. Take your time and study for your other classes and then come back and take this class next semester.” So we do this a lot. We have people who will stay in relationships because they’ve been in relationships for a while. So it’s like, I’m gonna hang in there when, you know, maybe it’s the end of the relationship. We’ll have people that put so much money into a car they’re gonna keep driving it. It might come to a point you just need to get rid of the car. So sometimes we’ll chase a course of action because we put so much money into it. The one thing I do want to point out, though, that’s important, is that sometimes sunk cost is very helpful. So if we’re in a job, for instance, that we’ve been in for five years, and we run into a really rough spot, we might say, “Well, I’ve been working pretty hard. I’m working my way up in this. I’ve got a lot invested. I’m going to stick this out.” A relationship’s another good example. Relationships do have rough times. And if you have been dating someone or married to somebody for a period of time, then it’s like, “I’m going to get through this because I have so much invested.” So a sunk cost can help you if down the road, it’s a good thing to stay in, but you’d be tempted to leave. College is a good example. I mean, junior year, a lot of people would quit, but it’s like, “I’ve already got three years in, I’m going to finish.” But at other times, we’ve got so much in, and it’s time to cut it loose. And this is the part where I love to tell people, when I was an undergraduate, I was a criminal justice major at first, and then I switched over and double majored with psychology, and then I ended up with a psych major and a criminal justice minor and a sociology minor, and I am one class short of having a bachelor’s degree in criminal justice. And every time I tell that, there are people in the group that says “You should just take that class,” and I’d say, “Why?” And they say, “Well, because you’re one class short, you’ve got all that invested.” And to that, I get to say, “At my age, I’m getting close to retirement age. I’m never going to work as a police officer. I don’t even think those credits would count. But even so, what I did is I got to a point where I thought, is it worth it for me to spend a semester to finish this when I don’t believe it’s anything I will ever do. I worked with a police department for a short period of time. At the end, decided I just didn’t want to go that route. There’s no reason to finish that.” But even listening today, if I tell you, in an entire bachelor’s degree, I just need one more three-credit hour class, the inclinations to say, “Well, you should take it.” That’s a sunk cost.

Rebecca: As I’ve been listening to you talk today and thinking about the things in your book, and I was also thinking about a lot of the things that some of our faculty panelists said at our graduate orientation, I was very happy that a lot of those things aligned. That’s always good. And one of the things that came up actually at orientation was the recommendation from one of our panelists to really think about some wellness things like sleep and exercise. And that’s something that came up and eating well, which is something that is in your book. Can you talk a little bit about some of the benefits of having adequate sleep and engaging in exercise that you talk about in your book?

Todd: Yeah, it’s funny, because when I’m talking to students, especially, if there’s quite a bit of material, there’s invariably somebody will say: “So if there’s just one thing I should really work on, we’ve talked about the testing effect, practicing over and over and quizzing yourself. We’ve talked about things like interleaving, where you mix materials together, or spaced learning, where you space it out. And for memories, is to practice over and over types of things, and to say, “Which one should I really focus on?” I said “sleep.” The best thing you can do to be better at learning is to make sure you have a good chunk of sleep. Not only as a restful person, do you process information faster, but while you’re sleeping, you actually consolidate memories. You could study all day long, all through the night. Take a quiz, and you’ve all done this at some point, do okay on the quiz, because that’s right during the learning period, come back a week later, or final exam, for sure, and it’s like none of it’s there. Well, it’s because after you learned it, you never slept, which allows your brain to consolidate it. We actually now know that different stages of sleep will consolidate certain semantic like meaningful things. A different stage of sleep will solidify and consolidate repetition, like movement, so the kinesthetic things. So the physiology of what your brain does is really important. By the way. I was just reading something the other day too. I can’t cite it here, because I didn’t know I’d be talking about it, but it talked about the correlation between low levels of sleep and later cognitive decline, because while you sleep, the brain kind of flushes out all of the toxins, so to speak, and all the stuff it doesn’t need. It consolidates the stuff that’s important. It gets rid of the rest. And if you don’t sleep, a lot of that becomes muddled across time. And so sleep, physiologically, is just an amazing thing for you. And so that’s a really, really important thing. Exercise is another one. Regular aerobic exercise is just good for the learning process. I had a student that told me one time… it was funny, we were chatting about this… and said he does the best learning during basketball season when they’re exercising all the time. Coach makes them sleep an adequate amount, because it impacts your play, of course, and if it impacts your play. It probably impacts your learning too, right? So the concept there is that he said he always did better during basketball season because he was sleeping better and exercising better. Eating is the same thing. We know that simple sugars are not as good as complex carbohydrates. There’s actually ways that we can process this in terms of memory, and it doesn’t take very much liquid at all. If you get dehydrated, a little bit of dehydration, you have cognitive decline. In fact, it’s funny, because if you talk to somebody who is a health professional and you say, late in the day, “Man, I’m just confused and really having trouble processing things,” one of the first things that will ask you is, “How much did you drink today?” So it does have an impact on that. So, adequate hydration, not too much, but the right amount there, good nutrition, sleeping and exercise, those are four ways right there that you can actually impact your learning of material in classes… oh, and your health. I’d like to point out too that it’s one of those things that’s got the best side effect there is: it’s really good for learning and it’s also good for living.

Rebecca: One of the key tips that came up from our students and our faculty was to actually schedule some of those to make sure you’re doing those things.

Todd: Oh, totally. Yeah, eating too, I have to admit for myself, especially when it’s hot out. I didn’t understand this when I was younger, but I can eat breakfast and work outside in the yard, come in, drink a little bit, go back out, sit down and read a book and do some stuff, and realize at eight o’clock at night’s like, “Oh my word. I didn’t eat today,” and so I’m suddenly hungry, but then it’s late, so you’re eating at eight, nine o’clock, which is not ideal. But if you schedule it while you’re busy, and stick to it. If you’ve got it scheduled, you’re gonna eat at one o’clock or noon, and the bell rings summarize where you’re at, go eat, and then come back and then pick it up from there. Because if you think “I’ll just finish this section and then I’ll eat,” there’s been lots of study through the years that says that’s not as good as stopping, noting where you are and then coming back for most people.

John: And one of the things you talk about in your book is scheduling. And you also suggest there that you include a little bit of slack, because things don’t always take the amount of time that you anticipate. Could you talk just a little bit about that?

Todd: Yeah, I kind of laugh about this one, because, yeah, people are hideous at estimates. Almost everybody is. But I will tell you, folks with ADHD are particularly bad at it. And part of the issue you run into with ADHD is it’s not just that you’re distracted, is that you’re thinking to yourself, I will do this task, it will take like, 40 minutes, and then I’ll go do this other task. When I got tested for ADHD, after three questions, like, we’re good, we know exactly where we’re at. Anybody who knows me says you don’t need any kind of test. We’ve known that for years. I wish somebody had told me 10 years ago. But the reason I mentioned this is that I’m working really hard at some metacognitive kind of process of estimating how long it’s going to take me to do something. I set off the block of time. When the timer goes off, then I think about: “What did I do? What worked well? What didn’t work well?” That process takes seconds to do, by the way, it’s like, okay, “did it work or didn’t work?” And then if it does work, I think about what I did to kind of set that up. And if it doesn’t, I think about that. But I will tell you that I will often start out with a to do list of 10 items, and I’m sure that I’ll be done by noon, and so by 6pm I’ve gotten through three of them, and I still have seven to go. So I put those seven on my to do list for the next day, and then, because I am a super genius… I always use that phrase whenever I follow it with something that’s not super geniusy… but I’ll take those seven items and I’ll just put them on the to do list with another 10 items, and then I got 17 items that I’m going to do tomorrow, because that’s going to be easy to get through them. I’ll just do them really fast, and then I get through like five things. As you practice at that, you can actually practice at estimating how long things will take you, put it into a schedule. I did this was an intro psych class. It was one of the most impactful things I’ve done. Is the teacher had us keep track of one week in one-hour blocks. Like, if you sleep for eight hours, you block off and say, I slept. But then if you read, you put a block in for reading, and while you’re awake during the day, he had us doing metacognitive exercises too. You estimate how much do you think you’ll get done? How well did you do? And then you just put a grade in the block. So if it’s going to be I’m going to read the to read this chapter in this two hours, and I get through half of the chapter in the two hours. And I’ve done this other stuff, I get like, a D. And what I found was that I’m very effective in the morning, from about 6 am until about 10 am I can cram through… oh, it’s just so good. And then I find out that from about 2 to 4 pm I’m just awful. I have enough extrovertness in me that if I’m talking to somebody, it’s fine, but if I sit down to read something or work at something, it doesn’t work. So I schedule these things in. So I’m kind of coming back to the schedule, but I schedule these things in, but also try to make estimates of how long things will take. I will finish with this, tell your students, or ask your students, I guess, at the beginning of the semester to keep track of, as they read the first couple pages in every book, jot down, after five pages, how long did it take you, and figure out the average amount of time it takes you per page. Because students don’t understand this, and they’re not good at estimating their time. And I’ve had students before who are going to tell me, for the final exam, they’re going to reread all the chapters. And I say, how long is that going to take you? And they’ll say, “I don’t know.” And I’ll say, “The exam’s tomorrow. What do you think it’s going to be?” And students will tell me that they thought they could read a chapter in an hour, or the whole book in like six hours. And then they find out that their average is eight minutes a page, and say, “Alright, well, at eight minutes a page, you’re not going to read a chapter of 35 to 40 pages in an hour.” But the point is, it may be that you have a novel in English class that you average one minute a page. You might have a physics book where you average 20 minutes per page, and you might have a philosophy book where you average five minutes a page. If you know that, you can make some estimates. And so talk to your students about making those estimates. But rolling that all together, kind of figuring out estimates on how long things are going to take, and keep working at it, and then using metacognitive strategies to think about when it’s going well and not going well. And I’m still working at how to do this.

John: And that issue of cascading to do lists, one of the things I found is that there’s a lot of to do apps that you can put on your smartphone, and when the list gets too long, just start another one.

Rebecca: That’s what I do. [LAUGHTER]

Todd: Well, there’s a great strategy that I really do like. You list the really important things as A’s and then B’s and then C’s. And the one strategy I read one time was great. It says, Put all your things on the list, put down your A’s, your B’s, and your C’s, and then delete all your C’s. [LAUGHTER] And so it’s that same kind of a thing as at the end of the day, they’re not going to get done, just pitch them. But yeah, we put too many things on the list. I’m trying right now. Actually, I’ve reframed it in my head, and I’m desperately trying to put down the amount of stuff that I think that I can get done by noon, and I still block off eight until 6pm but I put my to do list thinking this can get done at noon. If I get done at noon, you know what? I’m only gonna work a half a day. And when I’m better at it, I’ll probably put more items on the list. But for right now, yesterday, I still was working till 8 pm on my stuff that should have been done by noon. An another little tip I’ll give you… I’m getting better at some of these… I always make up my to do list now just before I go to bed, when I’m as tired as I can be during the day, because if you make a to do list in the morning when you’re feeling really full of energy, then you’re very optimistic. If you do it when you’re exhausted, it’s a little closer. [LAUGHTER]

John: I believe that behavioral economists refer to this phenomenon of underestimating the amount of time required for a task as the planning fallacy, and I believe that I’ve heard that the people who are most productive underestimate the amount of time required by more than other people tend to, otherwise they’d never undertake many of the tasks that they engage in. There was a Freakonomics episode on it that I just listened to recently.

Rebecca: In my design class, I have students keep time sheets for their projects and things, and that helps them become more aware of how long it takes them to do different parts of projects, so that if they were to take on freelance work or whatever, they have more of an idea of how long it actually takes them to do things, so that they could more adequately charge for work if they were to do that.

Todd: Rebecca, that’s a really, really important skill to teach them, because you could make up your mind you want to have a design company or a landscaping company, whatever it is, and if you’re hideous at estimating time, you’ll be out of business very quickly. When you explain to somebody, “Yeah, I’ll cut your loan on every week, and I’ll only charge you $20 because I think it’s going to take 10 minutes,” and then it turns out it takes two hours. You can’t stay in business like that. So I think that’s a great skill. See, we started out with the why don’t we teach these cognitive things early? This is one of those that you are doing, which is great. Teach them how to think about time.

Rebecca: Well, we could talk about really important things we could teach students all day long, but we do need to pay attention to our time, which we are not always good at doing. [LAUGHTER] So we always wrap up by asking: “what’s next?”

Todd: Probably just the last items on my to do list is next, [LAUGHTER] but right after that, I’m working on a couple things. So one of them is a book on lecturing and active learning. The research does not say that the active learning is more effective than lecturing. The research is very clear that says, when you combine the two, lecturing with active learning, it’s more effective than lecturing alone. So I’m working on a book that’s balancing the two. And then the other one is, I’ve really become pretty passionate lately about people that are not built for the way the educational system’s built. I love to use the phrase that education is kind of built for fast-talking, risk-taking extroverts. I’ve got that in the book, and actually in a couple of the book. And the idea is in class when the teacher says, “Could somebody tell me…” there’s hands going up before you actually get to the end of this question even from some people, individuals who think really fast, who are okay if they’re wrong, they’re taking a shot at it, and they’re extroverted in that they like to really have the energy to be around people. If you’re not one of those individuals, you’re actually at a disadvantage all the way through the educational system whenever something’s run like that. And so the other thing I’m working on right now is, how do we help the individuals who are shy, who are on the autism spectrum, who maybe have depressive episodes, individuals with food insecurity, housing insecurity, all these types of individuals who walk into a classroom and when they do really well, it’s because they’re doing so much more work. I should be careful. I don’t want to discount the people who are working hard in the classroom, but there’s individuals who are working way harder than other individuals, and there’s got to be a way to balance that a little bit. That’s what I’m working on now too.

Rebecca: Sound like really important, interesting projects.

John: And will those both be ready this year, or [LAUGHTER] will it take a little bit longer for those?

Todd: Probably Tuesday, I think… maybe Thursday. It’s interesting. I will say that I really love writing. It’s obvious, I’ve got several books out now. But I work with an editor. I have my own editor that I’ve hired now, and the reason I did that is because I’m not an editor. And what I used to do on books is I would write something and I would spend more time trying to edit it and get it to sound right than I was actually writing. And so the reason that I’m kind of cranking books out right now is I write it, I send it off to my editor, my editor reads it, comes back to me and says,” I like what you did here, but you might want to change that.” And then I just zip into the next thing. So it takes me about four months to write a book now. And so I would suspect that we will see one of those heading off… well, it won’t be too long.

Rebecca: Must be there on the early part of that list, that stuff before noon.

Todd: When I mentioned that keeping track of the day, it’s a really, really important thing. It’s because, yup, that’s exactly what it is. My writing time tends to be between 7am and 10am and then all my committee meetings are in the afternoons.

Rebecca: Prioritize.

Todd: Yeah, it’s like recording this show. Huh, look at this, 3:46 pm… Ha! Right, between two and four.

Rebecca: I did notice that. I was making that note,

Todd: Uh huh, I knew you were. Boy, we should do this sometime at 10 am You wouldn’t believe how fired up I am the.

Rebecca: Alright, but that’s when I prioritize my writing and doing stuff too.

John: We always enjoy talking to you, and we’re very much looking forward to talking to you about those next books as those approach,

Todd: I always love coming to this program. I get to sip my tea, which is always good, and chat with two of my favorite pod people.

Rebecca: Thanks for joining us, Todd.

Todd: Thanks for inviting me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC].

357. Inclusive Socratic Teaching

Socratic teaching is a primary pedagogical technique in American law school education. In this episode, Jamie Abrams joins us to discuss barriers this method can impose and strategies for a more inclusive approach to Socratic teaching.

Jamie is a Professor of Law and the Director of the Legal Rhetoric Program at the American University Washington College of Law.  She has published numerous books, chapters, and articles, including several on legal education pedagogy. Jamie is the recipient of teaching awards from Blackboard, the University of Louisville, and the American University Washington College of Law. She also co-founded the Brandeis Human Rights Advocacy Program at the University of Louisville Brandeis School of Law which works to advance the human rights of immigrants, refugees, and noncitizens.

Show Notes

Transcript

John: Socratic teaching is a primary pedagogical technique in American law school education. In this episode, we discuss barriers this method can impose and strategies for a more inclusive approach to Socratic teaching.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Jamie Abrams. Jamie is a Professor of Law and the Director of the Legal Rhetoric Program at the American University Washington College of Law. She has published numerous books, chapters, and articles, including several on legal education pedagogy. Jamie is the recipient of teaching awards from Blackboard, the University of Louisville, and the American University Washington College of Law. She also co-founded the Brandeis Human Rights Advocacy Program at the University of Louisville Brandeis School of Law which works to advance the human rights of immigrants, refugees, and noncitizens. Welcome, Jamie.

Jamie: Hello.

John: Our teas today are:… Jamie, are you drinking any tea?

Jamie: I am. I have a very simple green tea, but it’s in a special mug that one of my research assistants gave me that has a picture of the cover of a book I wrote and my research assistant worked on. So my tea is pretty basic, but my mug is pretty special.

John: And it’s on theme, because we will be talking about that book very shortly,

Rebecca: I have Scottish afternoon tea, despite the fact that we’re recording this in the morning. [LAUGHTER]

John: And I have Lady Grey tea today.

Rebecca: That’s a good choice, John. We invited you here today, Jamie, to talk about your most recent book, the one that’s on the mug, Inclusive Socratic Teaching: Why Law Schools Need It and How to Achieve It. Socratic teaching has long been a hallmark of legal education. Why has there been such a high level of curricular conformity in legal education?

Jamie: Yeah, it’s really kind of shocking, right? If we think for a second about what lawyers do. We hire lawyers to be innovators, problem solvers, creative thinkers, storytellers. So the idea that we educate in such a lockstep way is really kind of befuddling to me. You would think that schools, especially in a competitive market packed in fighting for these students, would find ways to differentiate. So I think personally, the US News and the larger than life role that the rankings processes have played over the years is one piece of the story. I also think it’s a lot of mimicry. So I became a law professor, and I started teaching how my law professors taught me, who were teaching how their law professors taught them. And also, I think there’s very little incentive to innovate. You have to sort of cross a certain threshold and be a quality and strong teacher, but at the end of the day, most of the incentives in academia generally, are skewed toward publication and other things, and there’s not a lot of incentive within our institutions. There’s no one to tell that you’ve innovated. There’s no one celebrating it or putting it on the website, but they’ll put your book on the website, and they’ll put your publications on the website. But I do think it’s ironic that the Socratic method was innovative at one point, that it was revolutionary to legal education, and now 100 years later, I’m not sure we can call it revolutionary anymore.

John: What is some of the harm done by this focus on Socratic teaching?

Jamie: Yeah.So first, let me just define Socratic teaching, just to make sure. So folks might define it different ways, but it’s typically a large lecture hall full of students, one professor situated at the front, and that Professor holds all of the power and engages in sort of serial Q and A with one student, than another, than another. So it might last the whole class with one student, some professors might do cold calls, where everyone in the room is sitting there scared, waiting to be on call, or others might take volunteers. And so this method of teaching is very distinct to legal education, and it has wielded the exact same criticisms, frankly, for 50 years, criticism that it marginalizes some voices and magnifies others. If you think about a classroom design, that’s literally true, like certain voices just carry dominantly on those microphones in our classrooms, others are harder to pick up, harder to hear. There are quantitative studies going all the way back to the 90s that male students, and white male students particularly, are much more comfortable engaging in this sort of intellectual sparring with the professor and other students might find that culturally dissonant to the way they would engage with the professor. It can be very abstract, sort of talking in the air about what the law is, which can cause students to sort of leave their identities at the door. We sort of pretend that the law is neutral, and while people in the room have been harmed by some of the legal rules that we’re teaching, and we don’t often acknowledge that, or we leave those students to raise the critiques directly. So those are some of the most concrete harms. But the other thing I would just briefly mention, those are the harms that I think are sourced to the Socratic method of teaching, this abstract perspectivelessness, sort of marginalizing hierarchy. But the thing that I’ve wrestled with in my work is: what are we protecting here? Are the rest of the students just thriving and loving this method of teaching? And that’s where I think it’s important to note that we have a real wellness crisis, in general, in legal education and the practice of law, with one in 10 students reporting self harm, one in six students with clinically diagnosed depression, one in four students with alcohol dependence. And so I sort of look at that data and like, “What are we protecting here? Why are we so afraid to innovate or try something new?”

Rebecca: Law school student bodies have also become more diverse. Has legal education evolved to support these more diverse student populations? It’s sounding like it’s not.

Jamie: It really is not, or the way that I describe it is we’re innovating around the curricular core. So it’s like we’ve bought a house and we’re doing beautiful renovations to closets and third floors and guest rooms, but we’re leaving the kitchen and the living room, where people spend the bulk of their time, untouched, and so we have beautifully more diverse student bodies, faculties, and our communities. But honestly, most of the reforms that have happened along with the diversification of who’s studying law have been more structural and less curricular. So we’ve popped up all of these support mechanisms, mentoring programs, pipeline programs, scholarships, affinity organizations, academic success programming, named chairs and all of these things are helpful. They provide structural support, but again, those are really incremental supports that risk giving the message to students that students are the problem, not the pedagogy. It’s saying, “Well, if any students are not thriving in this pedagogy, we’ll hire all these people to help support you through it,” instead of maybe reflecting that maybe the pedagogy isn’t working for everybody. So those kinds of support systems are helpful, but they’re sort of more harm reduction than transformation, and so our curriculums have not changed dramatically, other than seminars and ad hoc things like that.

John: Going back to the pedagogy a bit, one of the things that has been emphasized quite a bit in terms of inclusive teaching more broadly, is the importance of low-stakes formative assessments. But it sounds as if that has not been the norm in at least the core curriculum in law schools. Could you talk a little bit about how assessment takes place and what type of feedback students get in their core courses?

Jamie: Yeah, and one of the reasons I’m so excited to be on your podcast is legal education is so far behind other fields in pedagogical transformations. And so just about 10 years ago, we moved to outcomes-based assessment, frankly, pretty begrudgingly and reluctantly, but the accreditation bodies moved us in that direction, and most folks did it kind of superficially. We’ve added some learning outcomes to our course, and that was about it. So a typical law school course marches through 14 weeks of these sort of high anxiety, intellectual sparring with your professor, and then it historically culminated in a one time only three- or four-hour exam where you sit and that’s it, and you may not ever get that exam back. You’ll get your grade, you will move on, and from that professor, you’re unlikely to be able to get too much feedback, maybe a sample answer, or maybe a little bit of a summary of what your professor was looking for. But along the way, this outcomes-based education comes along, and frankly, most law faculty adapted very simply by just moving that summative exam that was so stressful and opaque and anxiety producing from the end and then plopped a midterm in October or March that does the exact same thing. It’s just as opaque, it’s just as stressful, it’s just worth less. But we’ve not really adapted in very thoughtful ways to do more no-stakes, low-stakes assessment, or to really help students see an arc, like a course of development of how the Socratic method fits into any of this. Is this a form of assessment, or is it just a method of teaching? I don’t think we’re clear for ourselves what the heck we’re doing with this method of teaching.

Rebecca: So as you brought in more diverse students, and the pedagogy hasn’t changed, and you have these supports here,[LAUGHTER] and that you’ve talked about some things kind of changing along the margins. Has the curriculum changed around the margins? Is that where some of the change has happened?

Jamie: So there’s a lot more dynamic programming. We have many more seminars and interesting named chairs and folks in our institutions who are institutional experts. But that being said, like a typical course in contracts or tort law or civil procedure still looks pretty much the same. The book I use, for example, is on it’s like 15th edition. It’s been around for 80 something years, and so how has it adapted? It shoves some more notes in to offer some more diverse perspectives, maybe eliminating some of the masculinized language or the racialized language in a practice problem or a hypothetical students are wrestling with. So I would say that, generally speaking, our teaching has become less problematic. But that’s not the same as becoming inclusive or excellent.

John: And in your book, you also talk a little bit of how the curriculum reflects the dominant culture, which doesn’t bring in a lot of voices, just by the nature of the curriculum with existing case law. What perhaps should be done to bring in more voices so that the curriculum better connects to the more diverse student body?

Jamie: Yeah. So I think reimagining a Socratic classroom around clients, around skills, and around communities could go a long way. So the one thing that Socratic classrooms offer that is beneficial is repetition, which is very helpful for students learning a new skill. And every class, every time they walk in, they’re going to start with one case, then a second case, then a third case, and they’re going to do that in all of their classes. So there actually is a lot of repetition happening. So what if instead we transformed that repetition into beginning with a client who called a lawyer. Why did they call a lawyer? What did they want? What had happened to that person? What did the law say when the lawyer first researched the law and did that answer the client’s problem, or did it not? And in most of our cases, it didn’t, and so they had to go to court. So there’s just so much more we could do to pull the exact same materials, the exact same law that we’re studying, and pull it down into communities. Like, how is this actually affecting people? And if you think about it, we’re packing classrooms. I have 88 people in my class that starts next week. There’s so many perspectives in that class. There’s going to be international students, students from rural America, from urban America, students from every geography in the country, like there’s so many great moments to just hear from that. But so this serial, I’m going to talk to student one, then student two, then student three, we’re missing all of the great opportunity to bring in. So there’s this famous movie, The Paper Chase. And in this movie, the professor is a legendary Socratic professor, and he says, “You came to me with a brain full of mush, and then I’m going to turn you into a lawyer.” And I think that that mentality, certainly less toxic today, but there’s still this idea that we need to eliminate, eradicate, this idea that our students come with a, quote, “brain full of mush,” and every year we have a room of 88 people from all over who’ve lived different experiences, who are going to read this exact same material and think about it differently, and our classrooms could be so much more engaging and different every semester, and more student centered if we start to embrace that instead of sticking with this serial case participation.

John: And I do have to say, as I was reading your book, I kept seeing John Houseman, both in the movie [LAUGHTER] and in the TV series. A brilliant actor.

Jamie: I think he won an Oscar for it,

John: I believe so, yeah.

Jamie: I have mad respect for anyone role playing a law professor who could win an Oscar. That’s impressive.

John: But I think that series did reflect some of the stress and anxiety that that method of teaching created for the students.

Jamie: Absolutely, there’s this famous scene. I can’t remember where it falls in the movie, but a group of these all white male students get a hotel room, and they lock themselves in, and they take the TV out, and they’re just spiraling into madness as they’re studying for the exam, and they’re talking about murder on the inside, such that the housekeeping staff ultimately gets suspicious of what’s going on in there. But it depicts everything of the toxicity. It certainly doesn’t look nothing like that anymore. But that the same idea of just this brain full of mush and then this frenzied push to an all in exam is certainly still there.

Rebecca: I think, whether you’re in a traditional kind of law faculty or in a different kind of department that still embraces traditional practices, no matter your discipline, introducing more inclusive practices can be daunting, can be hard. What do you see as some of the first steps to make some change? Is it something that you do as an individual? Is it something that you start to do and try to make a departmental change? How did the seeds of change happen, do you think?

Jamie: Yeah, one of the things that I think is a first problem to be addressed is we actually don’t have a community of teachers who teach in this way. And that’s actually kind of surprising. At least in legal education, there are robust communities of folks who teach legal writing, for example, that’s one of my areas of expertise. And we have conferences, we have newsletters, we have our own journals about pedagogy. We have idea banks, we have listservs where we share ideas. And that happens in clinical education. And so here, this is sort of like a rogue industry of Socratic teachers, sort of working as independent contractors, doing their own thing. You walk into a law clinic, whether it’s serving veterans in a private school, in a rural setting, or whether it’s doing housing cases in Manhattan, there’s a certain pedagogy and there’s a certain set of values that drive the work that these clinical classrooms are engaging in. And so I think one huge part of this is we need to build some community for those folks. There are 1000s of law professors, and more than that. My daughter is in AP gov and did a Socratic seminar as part of her coursework as well. And we’re exporting it around the world. A lot of law schools are trying to design themselves in an American tradition. And so I think one huge step is community building and figuring out how we can share ideas and have a sense of identity among Socratic teachers. So I think that’s a big part. Second, I think we also have a huge problem in academia generally, where we bifurcate staff and faculty roles, and so we have staff in our institutions who are working daily with students struggling through our classes, struggling through the bar, struggling through job placement, and then we have faculty just marching on doing their thing, and there’s very little information sharing about what kind of shared experiences, because it can depersonalize it. It’s not my classroom that’s a problem. But if we could hear our students in general are struggling with cold calling, with too much reading, with general anxiety, and we could start to be thinking much more as a community. And then finally, I think one of the big issues is this Socratic classroom props up the entire structure of time management for faculty, specifically. So I’m coming off my summer. I get back to teaching next week. And if I’m being honest, I probably spent 90% of my summer working on scholarship, and 10% working on teaching. And even the teaching, I wouldn’t really say, is working on teaching, it’s working on just rebooting the administrative components of it, updating my syllabus, getting my course learning management site up. It’s not teaching. I’m not transforming, I’m not redesigning, I’m not innovating, I’m marching forward. And I use my summers to do scholarship. And so I think for folks who are interested in innovating, we have to make some space for it so that others can follow.

John: There’s been a variety of critical approaches critiquing legal education. Have they had much of an impact?

Jamie: We have 50 years of scholars across a robust group of adjacent fields of scholarship, from feminist legal theory to critical race theory to LatCrit to queer theory. All of these critical perspectives have different methodologies, different agendas, and the one thing they all align on is naming the critiques of the Socratic method. And so I think, if I’m being reflective, these scholarly communities have had huge effects in building out that seminar programming, making more diverse hiring, and in the law itself, like how we think about what the law should be and scholarship, but at the end of the day, inside of the curriculum, I think the change has been a little bit more incremental, a little bit more add on. And so that’s kind of stressful for students too, to just shove in some extra readings here and there, but it’s not integrated in the book, and it sort of feels tangential or ad hoc. So I don’t think we’ve done a lot. Again, we’ve kind of done it around the core, and there’s great things happening. But I personally think that every person about to enter the legal profession should be thinking about it in more inclusive ways, of how the law affects communities. And just bluntly, every law is an expression of power, and somebody is going to benefit from that law, and someone is going to be harmed by it. And so it should be a part of training lawyers, not just seminar enrollment with your electives.

Rebecca: It’s really powerful just to just even think about that. I’m thinking about what you were saying about needing to shift from a 90/10 to a different kind of percentage, and how you start to make that shift if you also need to start forming community when a community doesn’t actually currently exist, because some of that time would need to be allocated to the community too. So it’s not as clean as a 90/10.

Jamie: That’s exactly right. And so, for example, I’m really excited since this book came out, I’ve done a couple of talks with faculty. Teaching is so vulnerable. Reading course evaluations, you have to be in the right mental space to open those. It’s hard, it’s searing, it’s vulnerable, just like our students’ experience when they open our feedback on their essays and their exams. And so I think here, if it feels like an attack and someone saying your teaching is outdated or your teaching is too traditional, it shuts people down. But if instead, we could bring in a speaker and look at a script of a typical Socratic dialog: what were the facts of the case, what was the holding of the case, what was the issue in the case, and critique it. That’s pretty darn abstract. It actually is abstract. How could we reframe this? And then the legal writing faculty in the room and the clinical faculty might say, “Why don’t you ask who hired a lawyer and why? Because those are the people who walk into our clinic.” And then the legal writing, people might say, “Why don’t you ask what legal sources that lawyer found when they researched on behalf of their client?” And so I think there are just cool ways within our faculty where we can all be learning from each other, but it does, I think, have to be a bit community based, otherwise it just feels too personal. And so one of the things I talk about in the book is this kind of paradoxical opportunity that COVID created, because the very idea that 180 something law schools and all of these faculties, in a flip of a switch, changed how they taught, and it wasn’t right, it was frenzied and it was flawed, but it was necessary, and we did it. And so, similarly, I think here, if it feels less personal, and it feels like we have to do this for our students, because one in 10 of our students is engaging in self harm, and one in four has alcohol dependence, like we can make some changes. I think we feel like we’re doing something together, instead of feeling like we’re attacked by someone critiquing our teaching.

John: You advocate inclusive Socratic teaching, so you’re not suggesting that Socratic teaching be eliminated. What aspects of Socratic teaching would be kept? What are the benefits of it that you’d like to see persisting?

Jamie: Yeah, and first I should say advocating might be a strong word for keeping Socratic method. I think I just approach it from sheer pragmatism… that change, you can’t bludgeon change. Change is hard. Faculties are not the most change oriented groups of people. The critiques are fair, and I think people actually agree with those critiques, but I do not think that the critical mass is there ready to upend the Socratic classroom and reimagine legal education. And so I sort of start from that premise and then say, “Okay, well, what could we do within this?” And so I kind of pitch my thinking on this as we should keep blowing the roof off of legal education, reimagining how we think about teaching. But while we’re doing that, which takes some time, we have to raise the floor and just raise our standards. So the things that I suggest are that we come up with a set of shared pedagogies that shape a Socratic classroom, and I propose that those frame Socratic teaching around techniques that are student centered, skill centered, client centered and community centered. And so my thinking with that is changing books is hard, changing testing techniques is hard, changing your teaching notes is hard, but if we could just reframe our existing dialogs using our existing materials, I think we could go a long way to diffusing that power role that the professor plays at the front. And frankly, it’s kind of liberating to hand a lot of power to the students. It takes pressure off the professor. It’s not this intellectual sparring, it’s a community of soon to be lawyers working together to talk about cases and what happened. And so I think it’s just a much more achievable method of reform. But in bulk, given how much of the time our students spend in these courses, I think the results could be transformative.

Rebecca: It seems like one of the key things that you’re promoting is the idea of the connectedness, that it’s not abstract, and that it’s connected to each individual, and that there’s a way for them to see how it would impact their lives, or the lives of their individual communities, or the communities that they’re a part of.

Jamie: Yeah. And I think the wealth of scholars who are writing about wellness in the profession are really great on this topic, which is, if we were to try to source where is this depression coming from… what’s happening? Law students enter legal education with similar levels of happiness and healthiness to their similarly situated peers, and then something’s happening in that first year. And I think that’s exactly what it is. It is the loss of community. Students are afraid to come into office hours, that someone else might hear their questions. And it varies at different schools how competitive and how individualistic the pursuit is, but when I think about my time practicing law, it was not individualistic at all. It’s sitting in a conference room hashing out ideas, throwing out ideas, brainstorming what to do, coming up with a plan B and a Plan C and a Plan D, and teams of lawyers thinking things through and the study of law is a lot of holeing yourself up in the library and just grinding it out, and maybe you might look to the people to your right and left as your competition. Certainly we need to do a lot more to help see that these are your future collaborators and colleagues, and you’re joining a profession that is a community, and you’ll rely on them much more in your practice than law school gives the impression of.

John: So shifting to more teamwork and collaborative work would be part of this revision. What other changes would you like to see in terms of how classrooms are functioning in a more inclusive approach?

Jamie: I would say, first, we need to be transparent with our students. If we’re using the Socratic method, why? What is happening, and it’s very opaque. So for example, if I were to propose one change to the most cynical or incrementalist of teachers who’s sort of skeptically considering this. Very simply, just tell your students what you’re doing with the Socratic method. Why are you using it? How does it prepare you for the exam? How does it prepare you for lawyering or the bar exam? What is it doing? And that can just be helpful. That’s a first step. Second. I think it’s helpful to start to think about pulling out of our Socratic methods. More clients, community, and skills. And then third, I think we have to tether this to a repackaging of our exam strategies as well. So what we’re doing in the classroom should be a form of no-stakes, low-stakes readiness for what you’re going to ask students to do on the exam. And so similarly, if we can start to ground our exams in more purposeful. So many of our exams are very abstract. You’ll give students a bunch of things that happen. I teach tort law with like personal injuries. So there’s a car accident, and then someone on those college campus scooters leaves it, and someone trips on it, and all sorts of sort of parade of horribles happening. And then we say: analyze. But who’s our client? Who’s paying us? What’s our mission? What was our community? Where did this happen? We could just do more to align what we’re doing in the classroom with these exams. And I think a lot of that aligns with where legal education is heading anyway, which is toward more professional identity formation, more development of cultural competencies. And so I think this, it can fulfill a lot of pedagogical objectives.

Rebecca: I hinted towards this a little bit earlier, about there’s individual things you can do to start to form community around you, or to connect with people who might be interested [LAUGHTER] in pursuing some of these things as well. Those individuals may not be your colleagues within your department. They may be other folks that you’ve met at conferences. How do you start nudging the people around you, maybe, who need the nudging? [LAUGHTER]

Jamie: Yeah, absolutely, I think it’s important to note that Socratic teaching in its traditional form is really hard on teachers too. We’ve talked about how it’s hard on students, but I look back on my early years of teaching, and it is a constant debilitating fear that your students are going to catch you in something you don’t know, or a mistake you made, and that’s just an unrealistic set of expectations. I practiced law. I was a talented lawyer. I’m a talented professor, but I still know only a fraction of the material that I’m teaching. There’s still room to learn. There’s still cases I never had and areas I never practiced in. And so it’s a very stress inducing experience for the professor, too, and particularly the new professor. And so over time, it becomes very sustainable, but at the beginning, it’s actually debilitatingly terrifying,[LAUGHTER] and especially it’s because of that intellectual sparring vibe. You can picture the student who’s like, mission in life is to outspar their professor, and so, that’s just a terrible approach to learning in general. And so I think a little bit of this, more experience sharing of how teaching can be more engaging, right? I teach the same courses every single year, but if we’re teaching in more student-centered, community-centered ways, our students are changing dramatically, how they think about the world, how they think about learning, our communities are changing dramatically, and our communities are, frankly, more polarized and divisive than ever. And so what are we going to do about that? So for tort law, for example, we’re going to go to a jury. Which kind of jury in COVID Are you going to get? The jury that COVID is not serious, or the jury where COVID has the whole community on lockdown? It’s a totally different set of values and norms driving a community. And so I think that it can just be such a more engaging way to teach if we open up the idea that our students are going to engage with the material in different ways every year, and it also takes the professor out of it, like I try to teach in inclusive Socratic techniques, and one of the things that I think is the biggest benefit of what I try to do is I don’t have a line out my door at exam time of panicked students. I do review sessions, and people come to office hours throughout, but if you’re more transparent with your students about what you expect of them and if they know how you’re going to test them at the end, and if every class is a chance to practice that over and over, it also takes a little bit of this tension out of the grades that I give and the conversations that you have with students around exam time, they can start to support themselves. You can start to transfer. They can study in groups and take practice problems together. And so in some respects, I think it’s less fatiguing and it feels less high stakes, because I can’t have all the answers. I have my lived experience. But when someone else engages on a topic in the class, the other folks in the class might have a dramatically better answer than I ever could have thought of. So I think just being a little bit more vulnerable and raw in like talking to each other about what Socratic teaching is actually like, especially in these first years, and I should mention especially for women professors and professors of color, because we have this image and this bias in our head of what a law professor looks like. And so if you stand there with a quieter voice, a smaller frame, if your voice doesn’t carry in the same way across the classroom, if you’re a person of color, there’s great series of talking about presumptions of competence, and how students on bulk don’t give a presumption of competence to their professors of color. And then you imagine that intellectual sparring and what that’s like when you have someone who looks like what you thought your law professor would look like, and someone who doesn’t. It’s a dramatically different experience. And so that was a very long answer, but building community and more honesty about what teaching in this tradition feels like.

Rebecca: So I feel like you might have some folks who might just say, “but Jamie, I’ve got my habits and I’m tired [LAUGHTER]…

Jamie: Yes, yes. [LAUGHTER]

Rebecca: …and I don’t wanna,” but I feel like those same folks might buy in if you say it’s really important for the new people, or for the diversification of the field, to make a little bit of room for these new folks and to establish some of these other kinds of patterns so they don’t get the pushback, and that they don’t have some of this stress. A lot of times, I think some of our seasoned faculty actually are really willing to budge a little bit so that some of the people that they really want to see succeed can, if they understand why those people can’t succeed. Sometimes you have to be really explicit about why they can’t succeed though, they just don’t get it.

Jamie: Yeah, absolutely. And one of the things that I think is important too, is, despite everything we’ve talked about in our time together, about the critiques and the harms, course evals are great for these professors, right? Like these are beloved professors. They’re revered, people aspire to be in their classrooms. There is tremendous peer pressure in teaching to conform. So one of the things I talk about in the book is, at a minimum, we just need to stop putting Socratic teaching on a pedestal and making it immune from accountability, even if you don’t want to change. Just stop pretending that your classroom is the default, and just make space that there could be other classrooms. And because what’s happening is anyone who wants to try to innovate or approach it in a different way is going to be evaluated by folks teaching in that traditional perspective. So there’s some room for academic deans to give some structure to what are we doing when we do peer evaluations, it’s not mimicry and conformity, like we should have some objective criteria that could be fulfilled in any number of teaching techniques, but there’s a lot of just this conformity. And so I think making some space to liberate peer evaluations, to rethink the questions we ask on our course evaluations, like the dominant questions on a law school course: “Was your professor knowledgeable? Was your professor available? Did your professor handle difficult topics well?” It’s all Professor centered, instead of, “Did you feel welcome in the classroom? Did you have the tools you needed to succeed?” Even our questions could be more student centered to start to measure things from the student perspective instead of let’s just assess whether you thought your professor was smart enough, which is an incredibly narrow and subjective and infused with bias way to approach it.

Rebecca: You mean like a question, like, “Did you receive feedback throughout the semester?”

Jamie: [LAUGHTER] That’s a great question. That would be a great question. One quick anecdote I would love to mention. As I was working on this book, I had the chance to present… every year, there is a new law professors conference that takes place in Washington, DC, run by the American Association of Law Schools. And so, I don’t know, maybe 150, 200 people come. These are the new hires. They’re entering. They’re going into academia all over the country. First of all, as a slight aside, everyone calls it the baby law profs, which already tells you that we are not embracing the idea that new people come in with new ideas. But anyway, putting that aside, I got the chance to do a session on the Socratic method there, and I would describe it as pretty raw, because in that room are people who want to change, like they’re at that moment, and they’re actually at a conference that’s doing some of the assimilation, and saying, “Let me teach you how to teach in the Socratic method,” like, “let me teach you how to handle exam writing.” And so that kind of tells you that there’s just this assimilation process, and on the topic of the Socratic method, it was quite raw, because many of those faculty, they’ve been in practice, and they see the flaws of it, they see the limitations of it, and they’re excited to innovate. And then slowly, we kind of dull that instinct and start to channel toward conformity in a way that I think is really problematic. That’s a great place where we could start. And let’s open it up, let’s hear from innovators. Who’s not teaching in a Socratic method? Let’s get them in front of these students and these new law professors, and let’s get a more diverse panel of what other techniques folks are using, what’s working, what’s not.

John: Following up with that issue of diversity, in your book, and you alluded to this just a few minutes ago, there seem to be some pretty substantial gender inequities in terms of legal instruction. Could you talk a little bit about that?

Jamie: Yeah, absolutely. And I honestly would love to talk about it a little bit more from the perspective of personal narrative, because I think that will tell the story quite representatively. So I’ve been in teaching for 17 years. I have two children, and I’ve had zero maternity leaves. One I was lucky enough to have in June, and so it worked out. The second was born in September… didn’t work out… and I kept teaching through that semester. I was a term faculty member, as many women faculty are, so I was not full time, not eligible for FMLA, and I hadn’t been there for a year. And I remember talking to my academic dean. I remember coming home and being like, “he’s gonna let me teach.” And I was so grateful that I was gonna be allowed to come in after giving birth and keep teaching. And I look back on that, and I’m like, “Oh my gosh, there are so many layers of problems with that….” that I actually sincerely felt that, and that I was so grateful to just have a job through that. And so anyways, so I was in legal writing, which is dominated by women, as is clinical faculty, and really limited in its opportunities for tenure-track positions. And so I ultimately went on the tenure track, and you have to be geographically mobile often to do that. And so that landed me in the middle of Kentucky, in Louisville, Kentucky, from Manhattan, and my spouse was a financial services guy working in Manhattan, and somehow did not seem as excited about Louisville, Kentucky’s financial services market as I was, and so we ended up living there 10 years, and seven of them he commuted. Seven years of solo parenting. And like when you start to add that up, schools get out at three o’clock. You get the kids on the bus, you get to school. How many conferences is that? How many more papers could you have published? How many more networking dinners and Bar Association events or award if I were able to be a full time… I was a full-time professor, but solo parenting, just to have a job, not to rise in the job or thrive in the job, but just to call myself a professor. Quite candidly, it nearly destroyed our family. Seven years of commuting is financially taxing, it’s emotionally taxing, it’s very hard on relationships, and that experience is so typical. I could name dozens and dozens. We are not great at spousal hiring. We’re not great at childcare. We’ve got lots of issues and just lots of stunted opportunities. And then the last thing I’ll say is all those things we talked about, the mentoring, the support programs, the affinity groups, those are often led by women and professors of color. And so there’s a piece of legal scholarship that I really like that says the women professors do the housekeeping at home and the housekeeping institutionally. And so women faculty and faculty of color doing disproportionate amounts of service, particularly the kind of service that isn’t really counted very well. Certain kinds of service are quite well distinguished and coveted, but mentoring students can kind of fall off the grid, and being the person that people just come to for support is not really tracked, but that’s time lost. So those are some of my personal experiences with it. I think the most raw example of it is I will never forget the day I found out I was awarded tenure. We had multiple days of the meetings, and so I didn’t know which day would be the day my package would be considered, but the day we found out my spouse was out of town, and there’s a tradition of a champagne toast at the end for folks who were just awarded tenure. And I couldn’t be there for the champagne toast because I had to go home, and then I went home to solo parenting, and I just remember I was very upset about it, and I just remember feeling like that’s the stuff, that community and that I was there and I was thriving, but I was not able to engage in the ways that we would want our faculty to under those circumstances. That was a raw, dark note to end on.

Rebecca: Those are the realities, I think, of a lot of complexities of higher ed that we’re still trying to untangle.

Jamie: And I think the thing that excited me so much about you inviting me to join the podcast is we don’t talk very much across schools. We don’t talk across fields. And legal education is pretty darn far behind. We have a lot to learn from peers, if we’re open to it. And so I think that these struggles are not unique to legal education, and other fields and areas are much more ahead of us. And so if we could just open our minds to more learning,= 360 from those around us.

John: And your book should help contribute to that, at least in the legal profession.

Jamie: I hope so. My favorite thing to do with the book is do faculty presentations that are conversations where, similarly, where I begin with the presentation, and then I’m out of it, right? Just like I’m describing we should be doing in our classrooms, where, what’s happening at your school, what could you be doing? And it’s a really rewarding feeling to see faculty talking among themselves about pedagogy and not being talked at.

Rebecca: …sounds like a great place to get to our last question, which is: we always end by asking, what’s next?

Jamie: Well, I think if I were talking to colleagues listening who are teaching in the Socratic tradition, I would say, start with something. We often set such high goals for ourselves that it feels like too much. Just pick a couple of things, the transparency, try a different method of assessment, or talk to your colleagues about what methods of assessment might be more helpful to give feedback to students. My first message would be to start with just a couple of things and just see how it feels. We don’t need to rewrite our books or rewrite our syllabi, but I also think there’s a lot of institutional work that needs to be done. I think this grassroots and this coalition of the willing is one strategy. But also, I would love to see more leadership at the top of academic deans and institutional structures, just doing a little bit more. Like, what if we promoted in our marketing some of our great teachers? What if we could do some spotlighting on students talking about what they love about their great teachers. I think there’s a little more we could be doing to create a culture that values innovation in teaching. Lots of our teaching awards are kind of popularity contests. What if we had awards for innovation, where we start to incentivize in that direction? And so I think institutionally, there’s a lot of work we could do to start to set the tone at the top that this is something that actually matters, and we can actually differentiate in the kind of pedagogy we’re delivering.

Rebecca: Well, I hope your book sparks some great change in the spaces that you want the change to happen.

Jamie: Thank you so much. Conversation about teaching, I think, is the answer to the beginning of all of this. So I’m so grateful to be on this podcast and to see the work you all are doing in bringing folks together from various fields and traditions, and it’s really inspiring.

John: Well, thank you. I really enjoyed reading your book. It’s an area that we had not really talked about before, and it sounds like it’s an area where there could be some very productive changes made.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

355. Class Dismissed

Institutional racism in the form of redlining and unequal access to educational and housing opportunities have left generations of students without equitable access to higher education. In this episode, Anthony Abraham Jack joins us to discuss the challenges that first-gen students face and what colleges and faculty can do to reduce these inequities.

Tony is the Inaugural Faculty Director of the Boston University Newbury Center and Associate Professor of Higher Education Leadership at Boston University. Tony’s research has appeared in numerous scholarly publications and he is the recipient of numerous awards from the American Sociological Association, American Educational Studies Association, Association for the Study of Higher Education, Eastern Sociological Society, and the Society for the Study of Social Problems. He is the author of The Privileged Poor: How Elite Colleges Are Failing Disadvantaged Students and Class Dismissed: When Colleges Ignore Inequality and Students Pay the Price.

Show Notes

Transcript

John: Institutional racism in the form of redlining and unequal access to educational and housing opportunities have left generations of students without equitable access to higher education. In this episode, we examine the challenges that first-gen students face and what colleges and faculty can do to reduce these inequities.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Anthony Abraham Jack. He is the Inaugural Faculty Director of the Boston University Newbury Center and Associate Professor of Higher Education Leadership at Boston University. Tony’s research has appeared in numerous scholarly publications and he is the recipient of numerous awards from the American Sociological Association, American Educational Studies Association, Association for the Study of Higher Education, Eastern Sociological Society, and the Society for the Study of Social Problems. He is the author of The Privileged Poor: How Elite Colleges Are Failing Disadvantaged Students. His new book, Class Dismissed: When Colleges Ignore Inequality and Students Pay the Price is scheduled for release in August. Welcome Tony.

Tony: Hi. Thank you all for having me.

John: Our teas today are:… Tony, are you drinking tea?

Tony: I’m actually drinking water, but I was actually thinking about going to get the Bengal spice tea that’s in my cabinet right now.

Rebecca: That sounds yummy, maybe afterwards, huh?

Tony: Indeed.

Rebecca: I have a new tea John that I got when I was in Scotland. So this is a 1903 blend from Pekoe Tea Edinburgh.

John: Wow. Yeah, I saw some of the photos from your world tour there while I’ve been in the classroom here, so just rub it in a little bit.

Rebecca: I’m trying. [LAUGHTER]

John: …and I am drinking an Irish Breakfast tea..

Rebecca: that’s on theme

John: …from Wegmans [LAUGHTER] here in Durham, North Carolina.

Rebecca: So, Tony, we invited you here today to discuss your forthcoming book, Class Dismissed. But first, maybe it’d be helpful if you tell us a little bit about your own educational journey.

Tony: There are two things that I always share about getting to this spot of being an educator and being an academic is: I’m a head start kid and I’m a first generation college student. Those two things are central to who I am and the work that I do. If it were not for Head Start, I would have a very different educational trajectory. Head Start put me in a school across the tracks, literally, in Miami. I went to a school that I wasn’t zoned for, because that dividing line that was McDonald’s Street. I was supposed to go to that dividing line of McDonald’s separated West Grove from Coconut Grove, where the Bahamian settlers were who cleaned up behind the white, wealthier families that live on the other side, and the schools were night and day in many respects, from the resources that they had, even class sizes were a little bit different. It was just a very different place. And what’s interesting is, if you were to come to my office right now, I actually have a map of Coconut Grove, and it literally says Coconut Grove Negro district, and on this official city map, you actually see the line that was drawn by city officials to say where black people could and could not live. That was fundamental. Head Start, it didn’t remove me from that history, but it didn’t make me travel the same path that that history would force me to any other way. And I’m a first-generation college student. I’m the son of a security guard. My mom actually helped segregate her middle school that she then worked at for 31 years as a school monitor, and my brother is a janitor who actually works at my old elementary school. No one in my family extended learning beyond college, but they made it possible through everything that life could throw at us, that made it possible for me to go to college, and those two things fundamentally shape who I am. As a Head Start kid, I can’t help but think about being from Coconut Grove. Every city in America has their Del Mar divide, that street that separates the richest and the poorest. Thinking about like it is in St Louis, I live very close to ours, and my education trajectory looks very different from everybody else who I went to school with. So many people who I live next to had a very different trajectory. So those two things really speak. And I was very fortunate to end up at Amherst College for undergrad, and then I came over to Harvard University to pursue my PhD. And to be honest with you, I haven’t left school yet. Since the moment I entered through Head Start, I have not left school. It’s been a lifelong dream to be a forever student, and I’m living it.

Rebecca: I’m living that lifelong student trajectory as well. [LAUGHTER]

John: Me too. The history of redlining and dividing districts and federal funding and loan availability in terms of those redlined districts, which weren’t always labeled that way, but they were still treated that way in terms of the availability of FHA loans and so forth…

Tony: Exactly.

John: …for a remarkably Long time, led to some horrible inequalities in terms of school financing.

Tony: Yeah, how the United States did policy for transportation. When you think about I-95 they literally drove highways through neighborhoods and say, “Okay, black people live over here, white people can live over here”. Even, how we handled the GI Bill, even though it was supposed to be one of the most universal policies. One, it wasn’t universal because so many women were not available for it, and then yet, the racialization of it was also a major aspect. So yeah, between redlining and blockbusting. So actions that real estate agents would say they would purposely introduce a black family in a neighborhood selling a house at like 125% of its value. But then when those whites who did not want to have a black neighbor, they were quick to sell and leave, and neighborhoods began going from all white to all black in that very short period of time. And you think about the resources that left because of income differences and segregation, weighed down by racism with income, I was removed from some of that, because Head Start placed me at Coconut Grove Elementary, and the principal there, Don Beebe, said he gets to stay. Not everybody had that opportunity.

John: And you talked about some of this in The Privileged Poor, where you talked about some of the inequities in colleges based on the amount of privilege that students had. Could you talk a little bit about the thesis of that to help set the stage for your more recent book?

Tony: Yeah, The Privileged Poor came about because of two things, and very much personal experience, but in two ways, experiential personal experience, how I experienced as a student, but it also what I had to grapple with as a graduate student, because as a student, I was trying to figure out what was going on when I got to Amherst, because so many people had gone to boarding school like Andover and Exeter and Saint Paul, or these really, really ritzy prep schools like Hackaday and Sidwell Friends and Nightingale I had graduated from a prep school in Miami, but from Head Start to 11th grade, I was a public school student. I had a football coach who liked athlete students but didn’t like student athletes, and so I transferred schools my senior year because I didn’t want to be under that kind of regime. And then Amherst called that football coach and said, “Hey, do you got a student who can pass our test?” He said, “Yeah, we got a kid this year,” and the rest was kind of history. The reason why I tell that story is because when I got to Amherst, I discovered that my individual detour through a private school was actually a well established on-ramp to elite colleges because of programs like Prep for Prep, A Better Chance, the Wight Foundation (W, I, G, H, T), and all these other programs that place lower income and sometimes lower income students of color into boarding day and preparatory high schools. So I’m like, “Okay, this is different.” This is people come from all over the place. It’s not just the rich people who are coming from these ritz y places. But then I got to graduate school, and as a sociologist, I’m reading about these inequalities and cultural inequality and race and all that stuff, but no one who’s writing about education is speaking about these students. They kept speaking about students as a monolithic group, really like they were painting with one brush stroke, and I was like, “Well, they got half the story right.” A lot of the people who I knew did struggle with isolation and difference and culture shock, but the other half of students, roughly, did not, like I had friends who studied abroad in high school for a year. I had friends who had flown on private jets. I had friends who had experiences that I only saw on movies and TV shows, like Gossip Girl. And so I put my experience as an undergrad in conversation with what I was learning as a graduate student and say, “Hey, something’s going on here.” And what I learned is colleges were hedging their bets. They were getting their new diversity from old sources. They were increasing the number of lower income students, but they were not saying was that they were going to private schools to get those students. And my research show that one half of lower income black students at elite colleges like Amherst and Harvard are actually the alumni of those boarding, day, and preparatory high schools. And so when I was dealing with all of that, that’s what came up with the privileged poor. Privileged poor is an oxymoron. It’s about somebody having little economic capital but having high stocks of cultural capital. They had been to elite places before. They had traveled to Tulum, they had traveled to Martha’s Vineyard, they had traveled to Nantucket, all of these very, very ritzy places. And when they got to college, they were in their fifth year of boarding school, really, and I contrast their experiences to the doubly disadvantaged, those lower income students who don’t have that same experience. And the reason why it was important to me to draw out this distinction was because any program that we created to help students in college, especially lower income and first year in college students, it would miss the mark, because if we had these two modal groups, these two people who are on different sides of the coin, if we try to navigate right down the middle, we will miss out on so many students. And so I use the privilege poor and the doubly disadvantaged, kind of like the miners’ canary, I’m able to show when and how class matters in college, both symbolically through cultural capital, those taken for granted ways of being that are valued by colleges and universities, but then also economic capital. I’ll give you just two quick examples. One, who knows what office hours are? Who’s comfortable using them? And who knows how valuable they can be, not just for your present but also your future? Usually, the literature and other people say, it’s usually the wealthier students, people whose fam went to college, I show that it wasn’t just those from money who felt comfortable, but those who had experience that only money could buy, the privileged poor and the middle class were more comfortable, the doubly disadvantaged were not. They got access to the letters recommendation. They got access to institutional resources at a higher rate than their lower income peers. But when campus closed during spring breaks, and it was about how much money you had in your pocket, now how much cultural capital you have in your mind, it didn’t matter whether you were privileged poor or doubly disadvantaged, you went hungry. And so I was able to show how students’ trajectories to college shape their pathways through college in a new and novel way. And so The Privileged Poor allowed me to engage with policy as well as academics and reducing some of the inequalities on campus.

John: The Privileged Poor was written before the pandemic, but then the pandemic hit and you started working on this new book. Could you talk a little bit about what you observed there and how that led to Class Dismissed?

Tony: Yes. So The Privileged Poor was published in 2019 and I was very fortunate that in the years in between now and 2019 I got to visit a lot of college and universities that have begun working with them to change their policies, and to date, over 80 universities have changed their dining hall policies to account for food insecurity on their campuses, both from small liberal arts colleges to large public institutions and every institution in between. And so I was having this good rapport. And The Privileged Poor also focuses on life on campus. It tries to understand how class shapes students’ engagement with their peers, professors, and the policies that govern campus life. And one thing that kept gnawing at me was that a question that The Privileged Poor cannot answer head on because of what wanted to focus on then was about students’ lives off campus, especially their families in their neighborhoods, and how that continues to shape how they move through campus. Class Dismissed came about because I was asked to do a project about how black students were experiencing inequalities on campus, especially during the pandemic, and I said I like the premise, but to fully understand how inequality shapes how students move through campus, I want to expand who I’m speaking to and the questions that I’m asking with a particular focus: how does the inequalities off campus shape daily life on campus? And that’s how this project began. I was invited to do this study by William Julius Wilson, a preeminent sociologist, and I interviewed 125 Asian, Black, Latino, Native, white and mixed students to understand truly how COVID exacerbated the very inequalities that universities were ignoring, especially how structural inequalities like poverty, segregation, the lasting legacy of redlining and blockbusting, which we already talked about at the beginning of this conversation, continues to shape how students move through campus, and I was very excited and intentional about including more groups to study, because I want to show you just how different their experiences are, especially by inclusion of Native students, who are sadly one of the most overlooked groups of college students that we have, to be able to show not just the lasting legacy of redlining and blockbusting, but the lasting legacy of land theft and disenfranchisement, deepens our understanding of what universities need to do to prepare for these students. I know we probably want to get into it, but there’s something about understanding where students come from that really shapes our understanding of where they return to and when COVID shut down campus, even if the policies for these closures were correct, we show just how ill prepared we were to help students navigate that transition. And again, and I say this in the book, this book is not a COVID book. This book is not about COVID. This book is about the inequalities that existed before that were worsened by campus closures and the pandemic. Because I really wanted us to understand where students come from and how that continues to shape their life, both as a springboard into more opportunity, because in the book, I talk about students who were able to take on unpaid internships during the pandemic. Even as the pandemic shut most of the world down, crippled some countries to the brink of bankruptcy and collapse, there were students who were taking on unpaid internships at venture capital firms, compared to then being able to show well, yeah, lower income students also took on unpaid labor, but it took on a very different form, and yet new is I’m actually going beyond just saying that both groups of students took on unpaid internships or unpaid labor, I actually show that the skill sets that are used by lower income students to carry out the work that are thrust upon them often requires a higher level skill set than those used by those in unpaid internships who complained about the grunt work and the busy work that they were able to do. But here’s the rub, whose CV is going to say what? Whose CV is going to have the right title? Whose CV is going to have the right employer? Whose CV is going to have the right detailed job description? And whose CV is going to remain silent about all the extra work that they did in addition to classes, in addition to other familiar responsibilities? So that’s why I said, to me, I hope people take away from the book that yes, COVID had a different effect on students. But I really want us to pay concrete attention to the way in which so much of what I talk about existed before March 2020, before campus closed, and these processes were continued for generations because that black box of unpaid labor that happens for lower income students, that wasn’t created four years ago, that has already existed. But then I try to also offer concrete things that we can do to begin to understand this. Because I don’t know if you all remember, in the early 2000s the Common Application introduced the familiar responsibility question to understand why students are, for example, not part of Glee Club and key society and playing a sport. And some people like, “well, I work,” or “I take care of my grandparents, “and they saw this familiar responsibility question really changed the way in which admissions officers were able to get a texture of students’ life. I asked a simple but a fundamental question in the book, Why do we have this at the end of high school, beginning of college, to help students get into college, but we have nothing like it on the way out? What are employers missing by not understanding the unpaid labor that all students do, and not just those who go to familiar places, by venture capital firms or a senator’s office or some of the other things, knowing that the work they do is mostly grunt work that doesn’t give them any kind of true experience.

Rebecca: It’s interesting that all of these inequities have existed forever, and COVID did make things more visible, but that doesn’t mean that a lot of change has occurred. Can you talk a little bit about some of the ways that colleges continue to fall short?

Tony: Yeah, one of the things that I highlight in my work is, oftentimes, I don’t want to say the actions, but the happenings of the most overlooked offices on campus. And I’ll talk about like, you know, the Dean of Students Office and other places like that. And to give you an example, I talk a lot about academic leave policies in the book, and the reason why is because it often shows both the class biases and the, quite frankly, ignorance that colleges have about where students come from, and I talk a lot about where students come from, because we’re talking about universities that are reaching out and saying, “We want you. We want a student from every state and every type of community, like we want to diversify our campuses along all of these lines.” But are you ready for it? If you want it, you get credit for recruiting it. But are you ready? Do you actually know what students need? The reason why I talk about leave policies is because I exposed in the book the very class and paternalistic approach that we take to it. When a student wants to take a year off and do an internship or just take a year off just to travel, the only thing they have to do is email that Dean and say, “Hey, I’m not coming back next semester.” They fill out something, but it’s usually very light work. If a student is asked to take the time off, and a lot of people will say, “Well, it’s probably because they cheated, or that something like that,” it’s often not because of the work. It’s because they are struggling to manage the expectations of home and the reality of college. When universities tell students that they need to leave campus for academic leave, they almost act like parole officers and put a whole bunch of responsibilities on students, new responsibilities on students as they are going home. So even though a student is at their most vulnerable, many universities require students to work a full-time job and take class at a local university while they are away from campus. There are some universities that require letters of recommendation from that employer. Now to middle-class America, that may not sound so outrageous, but what if you come from a community where 50% population have jobs and the other half do not? What if you come from a community where you’re not only the first person in your family to go to college, you one of a few people in the entire community to go to college. Who is comfortable and familiar with writing the types of letter of recommendation to assess your mental ability when you are fighting to get a job at the local fast food joint or Walgreens or CVS or Duane Reade, because that’s all the jobs that you could possibly apply for in your area. If we don’t understand the geography of inequality, especially as it pertains to access to employment, access to schools, access to mental health counselors or individuals who can assess that, are we truly helping students when we’re giving them a task that can’t be done, not while they’re home? There are some students who had to leave home after they have left college to go to a place to be able to fulfill that because they live on the reservation and finding work in that way was impossible? What about our students who come from more rural communities where jobs aren’t as available as the apples on trees? There are so many things, and so I use students’ experiences to not only show how different those who come from privilege and those who do not, how different their college times are. But I also relate that to policies that we could actually change on the college campus to address the inequalities. It’s heartening that the University of Rochester, for example, has a program that they’ve had for years called the Take Five program, because they know that lower income students disproportionately have more disruptive life events that undercut their academic experience. They were like, you are admitted under this take five program, because we know that this time that you are investing in higher education is often going to be punctured with moments that leave you so broken and bruised you might take a week or two slower. And so I try to highlight those other things. And the reason why I talk about policy here is because, as I was doing this book, I was inspired by the work in The Privileged Poor, where I was able to get universities, for example, to change their dining hall policy. And so I sent out a theme for the book. I said, “Well, now that we know what we know, what are we going to do about it? What’s the point in just writing about inequality if we’re not going to address it? If students’ stories give the general scaffolding of policies that can be adopted, why not listen to them? Why not let that inform the universities that govern campus life?” I just want to bring attention to that because the fact of the matter is, as our communities become ever more diverse, by geography, by race, by class, we must pay attention to those sending communities in the same way how any immigration scholar will tell you country of origin matters because US policy differs, differs because of their relationship with each country. The same thing applies. Harvard and BU and Amherst have real nice diplomatic relations with Greenwich Village, with the Upper West Side, with Palo Alto, very great diplomatic relationships; from Coconut Grove, from Detroit, from parts of Brooklyn, from parts of Cleveland, St Louis, not so much. We don’t understand, and because we don’t have an understanding of that inequality, it’s hard for us to adopt policy that is receptive to it, because, again, to ask a student to take on a full- time job when they come from a community where jobs have been gone for a generation, is giving them a task that they cannot complete. The sad part about it is those requirements were not always relaxed during COVID19, some universities still required students to work during it. One student told me that she had to petition to get two part-time jobs to count because that was all she was able to find during the pandemic. And because what she come from, there weren’t a lot of jobs, and then she had to take on these two part time. Now, this is the same person who lived in a house with a family member who was on dialysis. This is the same person who lived with immunocompromised people, and she had to petition yet again and say, “Hey, I am afraid, deathly afraid, to go out to work. Please don’t make me because I’m fearful of hurting my family member.” She had to petition for it, because universities weren’t flexible enough to actually realize the consequences of it, or they were so rigid in their policy. And this book is not a book that is organized by, “Oh, this is policy, let me see the effect, right?” No, the book is organized by understanding how students dealt with the inequalities exacerbated by the pandemic in their families, finances, in the fault lines along race on campus. And when you look at the family dynamics and the consequences of it., you just see how it’s important to understand that students, especially lower income and first-gen college students, they’re not running away from home. They’re trying to get away from the things that make home hard. That’s why so many first gens are driven. It’s not because they don’t like home, it’s that they don’t like aspects of home that make life hell, whether that means having to do, as I talk about myself, whether that means having to do homework by candlelight because the power was out more often than not, whether it was not having your own bed or your own space to be, or enough to eat, those are very real thing. But I also want to highlight the ways in which university policies make coming from those spaces even harder, and to some students, they almost felt intentional.

John: So far, we’ve mostly been talking about institutional policies. But what role do individual faculty members play in this?

Tony: Individual faculty and staff are also important players in this, So, I’ll give you an example from the book. In the book, I show how universities have a segregated on-campus labor market where students are almost divided by social class in what jobs they go after and what jobs they get. What I show is that middle class students, or rather, students who are comfortable engaging with faculty were more likely to get research, teaching, and course assistantships. They were more likely to be in quote, unquote, those like life of the mind positions, the academic kind of heavy lifting work of the University. Students from lower income backgrounds who were not comfortable engaging with adults were more likely to be in manual labor positions like barista, grounds keeping, janitorial. That was happening long before the pandemic hit, and then when campus closed, and what I’m able to show in the book, is that lower income students who weren’t comfortable lost their jobs, those who were comfortable with faculty kept their jobs, and even increased their hours. I talk about this being directly about what faculty can and cannot do because so often assistantship jobs, whether it’s course, teaching, or research, those jobs aren’t even posted. Those jobs are doled out in office hours. Those jobs are people who are tapped for those positions. But if you are not in those spaces, even if you’re the best qualified because you know the material, you don’t get that job. In the end, what I show is the students who needed more were given less. The students who needed to earn more money to support themselves and their family were given fewer hours, less resources, and less support. The students who needed less, at least financially, were given more: more hours, more money, more support. It increased the disparity that what these jobs can do, and importantly, let’s then move beyond the pandemic. If you have a job when you have high faculty interactions, you not only get paid today, but you get a letter recommendation tomorrow. But if you have a transactional job where it’s pay for services rendered, that’s all you really get, is the pay today, there aren’t readily available opportunities for you to use your job to fully access institutional resources. Yet, if you work for me or any faculty member, there’s a greater likelihood that you will be able to access something from our networks, whether it’s, “Oh, I’m going to be meeting with the Director of HUD,” or “I’m meeting with this Director of a movie, they’re giving me three extra tickets. why don’t you come with me?” All of a sudden, you’re meeting people in different kind of ways. Or I. can put you on to say,”Hey, this person’s hiring next year. You should think about interning there.” These are just like very, very real opportunities that students who are connected to faculty members who are in the know or who can connect them with it, do more. They get more and so that’s why I want to challenge faculty. And faculty have great autonomy and greater access. One chemistry professor, for example, can hire as many students as an entire office like a dean of students. Some chemistry professors’ labs are 20 or 30 people strong, and half of them are undergraduates. So we have power within ourselves. I was doing a professional development program this weekend, and I met a Dean, and he said that he just changed the rule that if a faculty member does not list the job on the university’s website, they will not be eligible for a benefit that the college gives for hiring undergraduates. And I applauded them for that, because that is a step in the direction that I think we all should go. Because, as I also show in the book, when you make explicit the hiring process, lower income students from all backgrounds, whether it’s public or public school or private school, or their familiarity with elite environments, they’re more likely to apply or feel that it’s an opportunity that is available to them.

Rebecca: I think in this example and the previous example, you’re really highlighting the relational aspects that are really important for students to be successful in various ways, both on campus and in the next steps after campus. And you’ve really highlighted the idea of both in the example of the petitioning, and even knowing that petitioning is an option or building relationships with faculty, and even knowing that’s an option to get opportunities and highlighting the idea of making things more explicit. Is that the direction that institutions need to go in is developing policies that really highlight transparency, or are there other suggestions that you have for institutions?

Tony: I mean, transparency is huge, but it’s not just transparency. I think it’s also translation, because the more we leave to random connections, random conversations, or random moments of, “Hey, you look familiar,” or “Hey, do you take my class?” …the more we will see these class processes to continue, because students who are comfortable will avail themselves to more resources because they know how the college works, students who are not especially lower-income students think that hiring, grades, internships, fellowships, should be about the work, and yet, sometimes doing the best is not what gets noticed. It’s being present. It’s being in the room. I always tell the story of, when you apply for the Rhode to go to Oxford for study, you need eight letters of recommendation, eight. Who knows eight people, and not just know them, but who knows eight people who know them well enough to write true letters of recommendation for them, to endorse them for one of the most prestigious, if not the most prestigious, undergraduate award you can receive? You can be a 4.0 student, but if you’re a 4.0 student who has never been to office hours, who has never attended a faculty and student-faculty dinner, that has never worked for someone in their lab, do you have an advocate on campus for you who can say anything else, other than they were the top performer? But if you were a student who got like a 3.85, not quite 4.0, but still A range person, but you went to office hours because you didn’t do as well on your first exam, and you started going, and you develop a rapport with a faculty member, and you begin to work with that faculty member on their project. And then when that faculty member goes away for a week, sometimes you babysit their dog, or you become a member of their lab and their family, that letter is going to reflect that connection. But we leave so much in that black box of the hidden curriculum about how you should navigate college, and we almost never talk about employment. As a matter of fact, as I argue in the book, university takes a very hands-off approach to on-campus employment. We were like, “Just use this website or attend this job fair.” Job fairs are not neutral ground. We say “wear business casual.” What does that mean? As one student told me, “I don’t know what business casual was.” Some people may hear us talk in this conversation and be “Well, like, this is duh, like, it’s business casual. Get some loafers, get a jacket. Of course, you’re going to go and talk to a faculty member.” If anyone is having that thought right now, I would want them to ask themselves three questions: “What’s the highest level of education of the person who raised me? What job do they have? and how similar or different was my high school to the college that I went?” We have to remember, people with PhDs are 12 to 25 times more likely to have a PhD parent. Faculty members are like doctors, dentists, and the vets. It’s a hereditary position. So many faculty members have not only family members who have PhDs, but who are the children of PhDs. Again, through studying students’ experiences, through their families, finances, and the fault lines along race on campus, I was able to expose the way in which students move through campus on uneven and unequal grounds, long before the pandemic, and was able to chart how much rockier it got.

John: So, we’ve talked quite a bit about institutional issues and institutional policies, as well as some things specific to faculty. One of the things that happened with the pandemic, though, is faculty, often for the first time, recognized the extent of some of the inequities that their students were facing when they were seeing students who had trouble accessing wireless networks, or accessing Zoom, and struggling to share a computer, or being busy taking care of sick relatives or other issues, and there was at least a bit of recognition of some of these inequities. But I’m wondering if maybe faculty are backing a little bit away from that now as we’ve moved a bit further from the pandemic. A lot of people were a little bit more relaxed about deadlines and other policies and were a little more aware of the role of the hidden curriculum and so forth, but I’m not sure that that’s continued at the same level we’d like.

Tony: And that’s one of the reasons why I hope that this book doesn’t take us back to COVID, but it resurfaces the lessons that we learned through it, with some new ones, of course, because for me, I don’t want us to have this “get back to normal,” this post-pandemic normal, this new normal. The new normal is too easy. It will all too easy become a replication of what we had before if we don’t take these inequalities into consideration. If we don’t understand the way in which class shapes how students move through campus, and how race often exacerbates those differences, we just go back to what we had. We need to strive for something more like an equitable campus. We haven’t seen that.

Rebecca: Moving towards an equitable campus requires some hard work and change, and sometimes that’s something that people are really afraid of.

Tony: Yeah, I was thinking about something I said, it was like: “Simply following the impulse to get back to normal will only be a setback. If we return to normal, we will return to what was. We will once again be held hostage to the inequities that plague higher education and society alike. We will once again hold ourselves and our institutions back from being what we could be, what we should be. Let us not yearn for normalcy. I don’t want to be normal in the sense of the normal operating procedure. We need things to disrupt the normal operating procedure, unless our intention is to harm the very students who we are paying millions to recruit, retain, and graduate. If your intention is to do that harm, then you can go back with the new normal that’s basically going to be what we had part two. But that is not your intention, then you need to rethink and reframe many policies.”

John: In addition to the need for institutional change, are there some specific things that faculty can do in their classes to help reduce some of the effects of the inequities that our students face?

Tony: I’m a sociologist, and I already get in trouble for being as prescriptive as I am in the sense that “you’re too applied,” I’m like, “Well, again, now that we know, we know what we’re going to do about it,” but I don’t want to be so prescriptive that I limit the imagination of faculty. I’ll give you an example: in reading The Privileged Poor. I talk about the fact that we always say when office hours are. We almost never say what they are. And my work on office hours has spurred hundreds of faculty to begin to define office hours on their syllabi and on the first day of class. I want them to be similarly inspired. And here’s where it gets fun. Some people were in class 10 minutes early, and walked the class to their office to show people where they are. I never said to do that. I want to put people on the path to understanding the nature of America and how it comes to campus. We all don’t have the same streets to walk down. Some people gladly take a walk at night and stretch their legs. Other people know that that is a luxury that could cost them their life. And I say America, not saying that we are not global institutions, but we also know the larger context of the recruitment of lower-income students, especially in no-loan financial aid policies that are at nearly 80 no-loan schools in America. The vast majority of them only have no-loan policies for U.S. students. For me, that’s why I say America. And so I don’t want to be so prescriptive and tell fabrics what they need to do, but I do wish more people knew about redlining and blockbusting. I do wish more people knew about land theft and settler colonialism, and not just in the historical context, because we love to say that happened a long time ago, like Jim Crow laws were a long time ago. I’m like, “No, they weren’t.” I wish we understood the present-day manifestations of those inherently disenfranchising practices, because if we did, I think we would come to understand the reality that the greatest determining factor of an individual’s life chances in this country is a zip code in which you are born. In the land of the free, home of the brave, the place of unbound opportunity, if I know your zip code, I can predict how old you are going to be when you die, if you go to school, if you will graduate from high school, what college you would go to. Neighborhood isn’t destiny, but it almost feels sometimes that those who make it out took a detour that was a once-in-a-lifetime thing. There was a moment, there was a broken light that said, “turn here” instead of keep going and getting stuck into a different path. And so I just hope that this book spurs conversations on campus, not only to revisit some of the most invisible policies that have rather large impacts on student life, but also thinking about how we move through campus.

Rebecca: I hope a lot of faculty and administrators and other folks involved with our institutions read your book and really come together with creativity and have some interesting solutions or experiments to make change happen. We always wrap up by asking: “What’s next?”

Tony: What’s next? That is a great question. I told you I’m first-gen. I never got to do a college tour when I was in high school. I didn’t know what a college tour was when I went to the prep school, you know, I was around a whole bunch of rich people, and they were like, “Oh, well, we’re gonna take a week through the California schools and a week through the New England Schools.” And I was like, “whatever I can’t see online when I’m at school on a computer because I don’t have internet at home, it won’t be seen.” And so I always say that when I go and share the work with people, as I’m invited to campus. This is my college tour. And so one thing that I look forward to about what’s next is visiting campuses to share the book. And also I am knitting another baby blanket for my graduate advisee, who just welcomed his first child into the world with his wife. And so that’s what’s next for me, is the book tour and a baby blanket. I actually just went and got the yarn this weekend, and I chose a pattern.

Rebecca: That sounds really exciting, a nice balance.

Tony: Yeah. [LAUGHTER]

John: And we do appreciate your book tours. In fact, we very much appreciated when you visited Oswego after your previous book. And I know shortly after that visit, a lot of people stopped calling office hours, office hours, and started using other names and moved them to other places to make them a little bit more welcoming. So I know that was a fairly immediate effect of one of those college tour visits.

Tony: Actually, Oswego was my last in-person visit before campus closed.

John: Yeah, we closed just a week or two after that. I think. I remember that, yeah,

Tony: I think I came up on March, 7th or 8th, I believe, and campus closed on March 10. [LAUGHTER] It was my last visit. I got off the plane, I was like, “Yeah, this is a different world we are living in.

John: Well, we were glad you made it before everything shut down.

Tony: I am too

Rebecca: …extra memorable. [LAUGHTER]

John: Well, thank you. We really enjoyed talking to you, and we’re looking forward to sharing this episode and to reading the book, which should be arriving shortly, my pre-order went in a couple months ago.

Tony: Awesome. [LAUGHTER]

Rebecca: Thanks for joining us.

Tony: Thank you all so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

354. International College Students

International college students face cultural and financial challenges in addition to those all new college students face. In this episode, Peter Ghazarian and Hayley Weiner join us to discuss strategies institutions might use to support international students.  Peter is an Associate Professor of Higher Education Leadership in the School of Education at SUNY Oswego. He has worked in international education in the US, UK, Germany, and Korea. Peter’s work focuses on higher education, leadership, public policy, multiculturalism, and human migration. Hayley is a graduate student in the Higher Education Leadership program at SUNY Oswego.

Show Notes

Transcript

John:International college students face cultural and financial challenges in addition to those all new college students face. In this episode, we discuss strategies institutions might use to support international students.

[MUSIC]

John:Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Peter Ghazarian and Hayley Weiner. Peter is an Associate Professor of Higher Education Leadership in the School of Education at SUNY Oswego. Peter has worked in international education in the US, UK, Germany, and Korea. His work focuses on higher education, leadership, public policy, multiculturalism, and human migration. Hayley is a graduate student in the Higher Education Leadership program at SUNY Oswego. Welcome Peter and Haley.

Peter: Hi.

Hayley: Hi.

John:Our teas today are: … Peter, are you drinking any tea?

Peter: I actually am currently drinking some tea. For the summer, I would say my go to is unsweetened black tea, and that’s what I have right now. It’s hard to get away from it.

Rebecca: It’s perfect.

John:And Haley?

Hayley: I’m drinking chai today.

Rebecca: Nice.

John: …and Rebecca?

Rebecca: I am drinking a Scottish afternoon in anticipation of my upcoming trip in two days.

Peter: That’s exciting.

John:And I am drinking an Irish Breakfast tea, which is in the same general neighborhood.

Rebecca: I’m going there too. We invited you here today to discuss your research on the experience of international students in US education. Can you start by helping us understand the current landscape of international students studying abroad in the United States? Like, who are the students? Are they from particular regions, levels of affluence, et cetera?

Peter: Yeah. So I’d say there’s been kind of a sea change in the international student population in the United States since COVID. Prior to COVID, I would say it was predominantly dominated by middle- and upper-class students from China. But then with the trade war, and then with the COVID outbreak and the zero-COVID policy in mainland China, it led to a lot of diversification of international students coming to the United States after COVID. And so we are starting to see a lot more students from India in particular, and also from Sub-Saharan Africa, especially in Nigeria. A lot of students coming to the United States from Nigeria.

John:And could you talk a little bit about the research project that you’ve both been working on?

Hayley: Yes, so Peter and I were intrigued by this idea of why international students come to the United States and what their experiences are in terms of seeking employment. A lot of international students express this notion of wanting to gain either internship experiences or post-graduation employment, normally in the form of CPT or OPT and Peter and I are very intrigued, as these students often express positive and negative experiences that are unique compared to what their domestic peers may experience when trying to gain employment of that kind.

Peter: And we’re also kind of interested in these competing personal versus professional elements or aspects to their experience, where they have kind of this imagined experience that they’re going to have as international students. And then there’s this reality and these economic demands that come upon them that they might not have anticipated. And so there’s almost a direct kind of butting of heads there between those two aspects of their time abroad.

John:You mentioned OPT and CPT. Could you define those terms for our listeners?

Hayley: Yeah, so CPT stands for Curricular Practical Training, and this is a work authorization for international students while they’re maintaining their student status, so they can only participate in CPT for employment that is directly related to their major, and while they are in a current student status. OPT, on the other hand, stands for optional practical training, and this is specifically for students after they graduate. And traditionally, students will go for a full-time job, but students can do either part time or full time or even internship. But OPT is specifically for their post-graduation employment.

Peter: And so for a lot of students, that becomes this period where the clock is ticking down if they want to find visa sponsorship through an employer. And so many students see this almost as kind of this grace period after they finish their program to try out with an employer or employers, and hopefully be able to win their support to stay in the United States as a full-time employee there.

Hayley: Yeah, and to add on to what Peter said with this idea of a clock, specifically with OPT, with students finding their post-graduation employment, there’s a lot of federal rules. For example, the latest date that they can start their OPT is 60 days after their graduation date. And as we know, navigating the job market in the current way it is, it’s very difficult for domestic students, international students, or any student in general, to find a job within a specific, constrained timeframe. And international students don’t get the privilege of looking for jobs outside of their major or maybe just taking a job that can earn them a little bit of money while they seek something that is more up to what they’re looking for in the long run. So keeping in mind that there are these federal regulations about the days and there’s this constant pressure sitting on them to find the jobs, even while they’re a student and post graduation.

Peter: Once students become aware of that, it’s almost like the specter that’s haunting them in all of their experiences, whether or not they’re in class or whether they’re building personal relationships with their peers, there’s this creeping dread of this clock ticking down and this need to find some security in life.

Rebecca: Those federal regulations impact the front end of the experience as well. Can you talk a little bit about some of the challenges international students face in even coming to the United States to study?

Peter: So international students… I would say the biggest struggle is to get that student visa on the front end. So past work that I’ve done related to destination choice among international students has primarily focused on country image and how that influences individual students’ choices. And what I’ve found in the past is that actually much more important than individual perceptions of a destination are the country-level relationships between a student’s home country and the destination, and so they’re much more likely to choose a place that has a good, constructive, positive relationship with wherever it is they’re trying to go. That plays more of a dominant role in deciding whether they go or don’t go to a particular destination. But then there are also other issues that they consider, such as financial barriers and regulations that can get in the way of them easily being able to access higher education within a particular destination. And for the United States, the big barrier, I would say, is demonstrating the ability to financially pay for the study in the United States. It’s for those who’ve had the experience, it’s a bit like applying for a mortgage, but instead of being asked to show that you can pay for it in the future, that you can already pay for the entire house before. [LAUGHTER] And so there’s a lot of financial gymnastics that people go through to try and demonstrate that, even though it is a big ask for them and it can be very difficult for them.

Hayley: Yeah, to jump onto what Peter was mentioning about the financial aspect, I currently work as a DSO in SUNY Oswego’s international student office, and a DSO is a designated school official. So I help students when they’re preparing for their visa interviews or when they’re getting their initial documents, such as a form I-20 to apply for their visa interview. And a very common occurrence is international students needing additional time or having struggles in terms of demonstrating the financial ability, and specifically, international students don’t get the privilege of applying for FAFSA or other United States federal aid programs that their domestic counterparts may get the opportunity to apply for. And so as a result, they have a larger sum of money that they may need to come up with up front, like Peter mentioned. And sometimes, even if they don’t receive the visa on its own, they still needed to come up with that money just to get the visa interview appointment.

Rebecca: And then my understanding is that, depending on where you’re coming from, getting a visa appointment is also a challenge …just scheduling it and making it happen in a time frame that makes sense to start school.

Peter: I’ve had many applicants, for example, from Ghana, who are really struggling with that. It’s just booked out so far in advance, it’s so difficult for them to get an appointment, and the students feel as though they’re very unlikely to be awarded a visa unless they’ve been given some form of scholarship or some sort of financial assistance with the offer. And so students are pretty desperate in that situation.

Hayley: A lot of preparation goes into it as well. Peter mentioned Ghana. I know India is a country where sometimes students are waiting one year or more. I’ve seen students wait years …plural… just to get their visa interview booked. And sometimes students are more eager and they want to have it sooner. And this does result in a different perception of whether they decide to come to the United States specifically to pursue their degrees, or maybe what degree they’re going to pursue, because they’re looking for that longer benefit in terms of financial benefits.

John:Implicit in this discussion is the notion, I think, that most of the students who are coming from abroad are interested in staying here after it, or at least staying long enough to get some training. What proportion of students are coming with the intention of staying in the U.S. after they complete their schooling, and what proportion are planning to return to their countries?

Peter: I don’t have hard numbers for that, but the sense that I get is that a lot of students are pretty fluid in their view of whether they stay in the United States. A lot of students are also willing to go to other third destinations that are English speaking, for example, like Singapore or Hong Kong, or places where English is the lingua franca, and they feel like they’d still be competitive in the job market, but they would be making what we might call a global middle-class wage, although some of the students we spoke to were interested in going back home after getting a bit of experience here, believing it would allow them to move into kind of an elite position back in their home country. In particular, I believe one student was interested in moving for a global energy company that was active within their home country. They believed if they had some energy industry experience in the United States, it would be pretty easy for them to transition into that type of position back in their home country, which is a position for an elite individual there.

John:Given the additional financial requirements for students coming from abroad, it would seem like most of the students who come in would be coming in from above average incomes in many of the countries of origin, which means we’re mostly providing some additional benefits to those who are already relatively privileged in their countries. Is that accurate?

Peter: And that is absolutely accurate. I have actually done a study that looked at whether or not international higher education exacerbates income inequality. In any case, it depends on the host or the sending destination. In some places, it is something that is seen for the elite, and it is almost something that’s required for them to return back to their home and get a position within a large corporation or within the government. It’s seen as kind of a stamp of approval having that international qualification. But in other places, what we’ve started to see is that when we do have people who are from a lower income background who participate in international higher education, it does contribute to intergenerational mobility, and that tends to be an improvement in their social class as a result of their participation. And it just depends on the place. And so in a lot of East Asians countries that are sending students, it is contributing to the inequality, whereas in Europe, especially Eastern Europe, that’s where it’s having more of a positive impact and contributing to economic mobility there.

John:So you’re working on the study of international students in the US. Where’s the sample drawn from?

Hayley: We focus on a phenomenological study where we focused on the shared lived experiences of international students. And just due to our resources, we did use a population of SUNY Oswego students specifically, and these students were a mix of both current students and recent graduated students. And to my knowledge, all of our students were a undergraduate degree level or recent undergraduate alumni.

Peter: There were students from India, Malaysia, Nepal, Bahamas, and Senegal. We did reach out to other institutions in central New York, to their offices of international students and scholars, to ask them to kind of spread the word and help to recruit but we got only a few responses to those initial entreaties, and then when we tried to schedule interviews with those folks, there was no response.

Rebecca: What are some of the challenges that international students face while studying in the US and adapting to the US educational system?

Hayley: I think a big overlooked challenge that international students face, that maybe others may not acknowledge, is they’re kind of starting over in a brand new setting. They not only have to get adjusted to the culture, but there’s some things that maybe people who live in the United States for their whole lives take kind of for granted. So, for example, international students, they’re permitted to have an on-campus job, but they don’t have a social security number. They have to go through the process of applying for a social security number. International Students don’t have bank accounts in the United States, and they can’t even pay for food their first couple days here because they don’t have maybe a card or anything to use at, like the grocery store, or they don’t have a SIM card for their phones to call their families back home. And I think there’s a lot of cultural adjustment that takes place, and students are kind of rushed into being acquainted with United States higher education settings, because not only do they have to deal with all of these cultural adjustments, but they have their schoolwork that piles up almost immediately after they arrive.

Peter: Yeah, and in terms of what we learned just in speaking to participants in the research study, what was kind of surprising or jumped out to us was that they are going through a lot of the same identity development phases that domestic students or local students go through but there’s these additional layers on top of those experiences that they’re having with having to navigate a new cultural environment and dealing with some of this economic pressure that they have on them. For some folks, they’re able to navigate that successfully, and they’re alright, even though it is very stressful. But for others, it is very difficult. For them. There’s always something to worry about, and it makes it very difficult for them to just relax and enjoy the moment, because there’s always some issue or concern that’s kind of lingering in the back of their minds.

John:In terms of cultural differences that they have to navigate, one of those, I think, would be differences in the way in which instruction takes place in US classrooms. Could you talk a little bit about some of the challenges that students face in adjusting to a college environment in the U.S.

Peter: I would say, from our participants, we heard mostly positive things. So the students themselves, coming from a younger generation, felt happy and excited for maybe more of a student-focused approach to instruction and more active learning in the classroom. They appreciated the opportunity to be able to interact with their peers, and felt that it really strengthened their communicative abilities. And so it seemed to be a very positive thing for the vast majority of students. They did mention that it was strange at first and a little difficult to adjust to it, but that overall, it’s something they enjoyed. It’s something they saw valuable as part of this attractiveness of US higher education in general.

Rebecca: Some of the conversations that I had with some international students recently in terms of the kind of classroom setting was the expectation of scaffolding, or the steps to complete assignments that maybe they had an experience in their home country, and not understanding that those things were required, or that those were different kinds of expectations that we have here. So it’s made me as an instructor much more aware of being explicit about what some of those differences might be. Are there other ways that instructors can support this transition into the US system?

Peter: I would say, looking at it almost from kind of this universal design for learning perspective, would be the best way to think about it is that the more explicit and clear you are in general about your expectations, it’s going to benefit not just international students, it’s going to benefit all of the students that are in your course. So making sure that you’re communicating your expectations about assignments in multiple ways, so that the most people can get it or can understand it is going to be really key here in just improving instruction for everyone. And international students will, of course, also benefit from this, but I think everyone can benefit.

Rebecca: So in addition to adjusting to the classroom and some of the adjustments to the U.S., what are some other changes that international students describe?

Peter: So, one of the big things that we heard about was personal change. So students weren’t just adapting to a new cultural environment, it was also having an impact on how they saw themselves and their idea of themselves. And so a few of the students talked about internalizing ways of thinking that they had originally felt were foreign or different or that they were uncomfortable with, but now they saw it as part of who they were, and that they had this interesting experience of moving between selves. So there was their home country culture self, and then there was their US culture self, and depending on the context, they had to be a little bit fluid in their identity and how they behaved and the way that they thought, but that that was an important part of adapting and excelling within a cultural environment is having that flexibility to be able to reinterpret yourself according to the rules of a different game you might be playing according to which culture you’re operating in.

Hayley: And it seemed like when the students kind of described this personal change or transition, like Peter mentioned, it wasn’t really a superficial change that they were describing. They were describing more of like a deeper engagement with the new culture that they were experiencing, and how this deeper engagement worked to reshape their thought processes and their perspectives, and how it was very transformative within their cultural immersion into US higher education institutions,

Peter: There were times when they were able to achieve that imagined or idealized experience of studying abroad that they originally had before they came to the United States. So I think one participant in particular talked about walking home late at night from a part-time job that they held on campus, and how they would never be able to do that in their home country, because it would have been unsafe for them to be walking alone by themselves, and just how they felt so free and they felt so independent that they started crying on the walk home, because they felt like they had achieved what they had dreamed or hoped for by coming to the United States.

Hayley: And that kind of ties on to this, like aspirational kind of rationale that Peter and I focused on on our research as well, where both personal and professional international students have this motivation to come to the United States because there’s dreams that they have. And like Peter mentioned, there was that student who just felt this sense of relief, or just a sense of satisfaction of being where they were at that point, being in the United States, and they felt like they were living what they were dreaming of when they were a child. And then we have the professional side, where they’re international students. I remember one student describing to us how she was thinking of being a doctor, and how in her home country, she knew that being a doctor and being a female doctor didn’t have as much room or level for success. And then another student from the same country mentioned wanting to be a geologist and how, in her home country, geology is not a very popular science, and that more engineering and doctors are from that home country, and how coming to the United States allowed them to have this diversified curriculum that allowed them to fit into these dream roles that they imagined when they were younger.

Peter: And so what we were trying to make sense of these different rationales for studying abroad. We weren’t just doing this like on our own or pulling this out of the thin air. We came into this with a theoretical framework that we adapted from a researcher called Fakunle, who’s based in the United Kingdom at the University of Edinburgh and Fakunle has put together a model for the different types of rationales that international students have for studying abroad. And it’s something that she put together speaking primarily to international students in Europe. And we wanted to see how well it held up while we were speaking to international students here in the United States. And we found that the model applied very well. And those four rationales are an aspirational rationale, an educational rationale, an economic rationale, and then an experiential rationale.

Rebecca: That transitions nicely into thinking about international students for an F1 visa and that OPT or that optional practical training that we talked a little bit about earlier in trying to find a job in their field or discipline immediately following graduation. What is the experience like for students trying to find these opportunities? I know from my own conversations with international students, it can be a struggle. Can you share what some of those barriers are that they might face in this job hunt?

Hayley: I think a very major barrier is that sometimes employers don’t comprehend or understand what it means to be an international student, and believe it or not, with OPT, if a student plans to use their one year of OPT, or in STEM students’ case, they get an additional 24 months of OPT, an employer does not need to provide sponsorship of any kind, and the student is eligible to work for that given employer, so long as that employment is directly related to their major. And sometimes employers get scared or worried about the idea of hiring an international student, either just they don’t have the resources to know what it means, or they’ve never done it before, and that acts as a significant hinder for international students. Now I know when we were interviewing students, there were a few students who mentioned that they were able to easily acquire a job, and it took a few tries, but they were able to get one before they graduated. And we’ve had other students who mentioned that it was much more difficult to get post-graduation employment, and after graduating, it became their full-time job to apply for a job, but keeping in mind that they only had that limited number of days to seek that employment added this extra stress that was very troublesome for the students.

Peter: Yeah, and I would say that it’s not just the employers that are ignorant of the rules that surround OPT, there’s also a concern that support services on campus beyond the international student office, aren’t always aware of these regulations or how they impact a student’s employability, and so students felt like they had to take anything that they heard from Career Services or an internship opportunity that they heard about with a grain of salt, because those are almost always developed or they’re communicated in a way that’s targeted towards local students, and it might not play out the same way for an international student. That opportunity might not exist for them. And they also talked a lot about how they avoided medium or smaller companies as a result of those companies’ lack of awareness or knowledge of OPD or CPT, and the belief that they would just kind of not even consider them seriously because they weren’t aware of how OPT or CPT worked. And so this was a source of pressure for a lot of students. They felt like they were supposed to be the expert of these policies, but they’re very complicated, and for somebody, for example, who’s communicating in a foreign language, trying to express this in a way that calms the nerves of a potential employer is not something that’s easy to do.

John:Might part of that be due to concern that when many new employees are taken on, particularly in technical fields, there’s a lot of on-the-job training that’s provided, which is costly to the firm. And might there be concern on the firm’s part that they might invest in workers who will not be able to get a change in visa status?

Peter: Yeah, I think that’s absolutely a legitimate concern on the part of a company. Interestingly though, it seems like those technical positions are the ones where the students tend to have more success in applying for the positions. And so students, for example, who are going into mining or into IT roles, those are the students who often have the most success in their applications for OPT even though those tend to be the positions that have a lot of on-the-job training that they are going to receive. And I wonder whether or not that’s because those are industries that are traditionally more open to recruiting from abroad, and so perhaps they feel more comfortable with working with these students who might need some sort of sponsorship in the future,

John:And also the fact that there are not that many students in the U.S. majoring in those fields, so that a very large share of employees in those areas are coming from overseas … fortunately, or we’d be in much worse trouble in our country.

Rebecca: It might also be tied to the potential three years of OPT rather than one that is often tied with those particular fields which are STEM designated

Hayley: Based on our interviews with students, as well as just my work in general, I noticed that students who are in STEM designated majors, even for their first year of OPT, before their STEM extension is even granted, they have a much easier time just getting their foot in the door or getting an interview, compared to students who are maybe in the field of like business or communications, and I think that longer extension of OPT could possibly have an impact on an employer making the initial investment into an international student as a first-time employee.

Peter: Right, because one year isn’t really enough time to know for sure whether or not you would want to keep somebody on, but three years, you know then, you know for sure whether or not this is somebody you want to sponsor or not. And so, actually one of our participants made a comment like, “I just don’t think a year is enough for me to prove myself to an employer. And so I don’t think I’ll be able to stay. I’m glad that I have the opportunity to gain that experience here.” But, that person, in particular, really regretted not being in a field that offered the longer OPT because she felt like that would have given her a better chance to be able to stay in the United States in the future. An interesting sort of side effect of that that I’ve seen, is that a lot of institutions are coming up with ways to get their programs designated as being in the STEM fields, in order to make it more attractive for international students. So I I’ve seen, for example, like a business analytics program, that they really focused heavily on data visualization and those types of things in order to promote it and get it approved as a STEM field degree program. But is it actually a STEM field degree program? I don’t know.

Hayley: It is actually, yeah, business analytics was recently added to the list of STEM designated degrees.

Rebecca: What could institutions do or advocate for to minimize some of the challenges for students as they pursue work opportunities?

Hayley: I think, while it may not be the sole solution, a really great solution could be showing international students how to market themselves. I noticed a lot of international students worry that maybe because of their international background, they may not line up with a domestic peer who studied in the United States their entire lives. But that’s not always the case. International students have specific traits such as being able to easily adapt to new cultural settings, or a lot of times they speak even multiple languages, and this is something that their domestic counterparts who are applying for the same jobs, they don’t often have that same trait. And so by being able to market themselves in the current or modern workplace or in the job market itself, I think that would be one way to minimize some of the challenges. I don’t see that as being one of the ways to get rid of all challenges, but it could allow international students to have an upper hand.

Peter: And I think a lot of student support services that are available on campuses now are in this sort of one-size-fits-all approach to supporting students, and that that can be to the detriment of students who are coming from non-traditional backgrounds who are participating in higher education here in the United States. So international students, for example, suffer from this. Non-traditional students might suffer from this. First-generation students might suffer from maybe an older approach to providing support services to students that in the past did serve well, but now maybe we need to be thinking a little bit more carefully about how do we individualize these services that we’re providing, so that we are at least aware of the different types of support that different students are going to need if they’re going to be successful transitioning from higher education into the workplace, and so international students would absolutely benefit from this. And I think that, in order to make this work, there needs to be a little bit more communication across offices on university campuses, because now a lot of these things are very siloed off from one another, and so it prevents, for example, an understanding of students who are receiving accessibility support in their work: searching for a job, for example, or students who are international, who are looking for a job, might have specific needs that they would benefit from if those offices were aware of what one another were doing and they were sharing information a little bit more openly with one another.

John:A while back, we did an interview with someone from the University of Miami who was talking about a program they had there to help prepare students from China for American classrooms in terms of the difference in structure and the difference in expectations, in terms of active participation and discussion. Is that something that perhaps more colleges could do to ease some of the transitions?>

Peter: So there was kind of a boom in those types of programs, in those gateway programs, they were often called, and they met with mixed success. In some cases, they were really helpful, and it helped students to adapt by giving them a soft transition period when they could adapt to the expectations of the U.S. university, before they went into a more high-stakes environment where they were getting graded for their actual content area coursework. However, some people were critical of this. They saw it almost as a money grab by the institutions who were asking these students to complete maybe 24 credits of coursework that wasn’t really going to apply towards a degree, and a lot of the times it was a conditional part of their acceptance into that institution. And so they got kind of a bad rap by especially international students who saw it as just paying for two more semesters but not really receiving a tangible benefit.

John:That particular program was a very short course, just more of an orientation to the differences in the culture of American college education. It was just a short program to reduce some of the shock and to help prepare people for the difference so they could hit the ground running without getting a bit behind in their work.

Peter: Yeah, I think that approach is absolutely positive and something that would be of great benefit to the international students, but again, if it has no credits, and then the funding for that type of program becomes very difficult, because it’s not sustainable in the long run, a lot of the time,

Rebecca: Is there a role that institutions could… should… play in establishing relationships with potential employers or regional companies that might make it easier for students to find placements after graduation.

Hayley: Working in an international office, it’s not traditional for international advisors to maybe reach out to employers on behalf of students, but if an employer were to ask a student a question related to their visa status, or maybe related to OPT or CPT, that would be somewhere where an immigration advisor on a campus can step in. Something recently that the SUNY Oswego International Office did, was we actually heard about students receiving lots of questions from their employers about what it means to be an employer of somebody who is working under CPT authorization or OPT authorization. And so our office actually took the initiative to make a specific web page for those employers for these basic kind of Q&A, frequently asked questions, that we do receive, and it’s actually had a very positive impact. And we’ve received a few compliments from some employers about how this did assist them, and they learned some new information about really what it meant to be a student on OPT or a student on CPT from the perspective of an employer.

Peter: And that more broadly, with all of these workforce development initiatives that we’re seeing take root across the United States, I would say that there’s an increasing expectation that higher education institutions are more directly involved in the transition of their graduates into the workforce, and that they are playing a role in making sure that students, regardless of what major they’re choosing, are being prepared for some sort of future within the economy. And I think that there is some resistance to that, because it challenges some of the foundational ideas about what higher education is about, or the purpose of a higher education institution. But I don’t think it’s something that higher education institutions are going to be able to escape. With the huge amount of money that people are investing in a degree there is an expectation that there’s going to be a return on the money that students put into that and higher education institutions are going to be held accountable if students are not able to be successful after pursuing their degree.

John:Are there any other things that you’d like to share about your study with our listeners?

Peter: The thing that really jumped out to me is as someone in the past who has done primarily quantitative research focused on international students, it was nice and refreshing to focus more on the individual lived experiences. And what really stood out to me was how similar the experiences were between international students and what is recorded in the theory about local domestic students in the United States, and so they’re having these same sort of identity development phases, or they’re going through a lot of these same sort of experiences. But as I mentioned earlier, it’s layered with these other things that are going on that really add a little bit of depth and complexity to the types of things that are happening to the international students here in the United States.

Rebecca: We always wrap up by asking: “What’s next?”

Peter: I would say this study focused on Central New York in particular and so the next logical step would be expanding it out further to other regions of New York State, or to other regions of the United States in general, to make sure that my model holds up according to other experiences in other regions of the country and in private and public and even parochial higher education institutions within the United States. So grasping at the diversity of experiences that exist within U.S. higher education, and making sure that that’s captured in the work, and then moving beyond just defining the problems and the experiences towards highlighting or spotlighting solutions to these issues or concerns that we’re learning about as we’re studying the experiences of international students here.

Hayley: Yeah, and to kind of add on to what Peter was saying, by bringing to light or highlighting or making awareness of the shared experiences of international students, because I think it’s very important to acknowledge that international students still touch just about every office on the campus the same way that domestic counterpart students would. And so this is not just something that maybe international student advisors or individuals who work with international students on a daily basis should focus on, this is something that kind of has stakeholders on all ends of a campus. And so by spreading the awareness or making others known of these challenges that international students face, I think solutions would become more prominent.

Rebecca: Well, thank you both for getting us to think a little bit more about our international student population.

Peter: Oh, thank you so much for having us, and we really appreciate the opportunity.

Hayley: Yeah, thank you so much.

[MUSIC]

John:If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

353. Beyond ChatGPT

Faculty concerns over student use of AI tools often focus on issues of academic integrity. In this episode, Marc Watkins joins us to discussion how the use of AI tools may have on student skill development. Marc is the Assistant Director for Academic Innovation at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers.

Show Notes

Transcript

John: Faculty concerns over student use of AI tools often focus on issues of academic integrity. In this episode, we explore other impacts that the use of AI tools may have on student skill development.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by

John: , an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Marc Watkins. Marc is the Assistant Director for Academic Innovation at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers. Welcome back, Marc.

Marc: Thank you guys. I really appreciate it. I think this is my third time joining you all on the pod. This is great.

Rebecca: Today’s teas are:… Marc, are you drinking any tea?

Marc: I am. I’ve gotten really into some cold brew tea and this is cold brew Paris by Harney and Sons. So very good on a hot day.

John: We have some of the non-cold brewed version of that in our office because the Associate Director of the teaching center enjoys that Paris tea so we keep it stocked pretty regularly. It’s a good tea.

Rebecca: Yeah.

John: My tea today is a peppermint spearmint blend.

Rebecca: Sounds nice and refreshing. I have a Brodies Scottish afternoon tea, and it’s hot. And it’s like 95 here. And I’m not really sure why I’m drinking hot tea in this weather. But I am. [LAUGHTER]

John: Well, I am here in North Carolina, and it’s 90 degrees. So it’s much cooler down here in the south, which is kind of nice. [LAUGHTER] And actually, it’s 71 degrees in this room because the air conditioning is functioning nicely.

Rebecca: Yeah, my studio at home… the one room where the air doesn’t work. So hopefully I don’t melt in the next hour.

John: So we’ve invited you here today to discuss your recent Beyond ChatGPT substack series on the impact of generative AI on student learning. Many faculty have expressed concerns about academic integrity issues, but the focus of your posts have been on how student use of AI tools might impact skill development. And your first post in this series discusses the impact of AI on student reading skills. You note that AI tools can quickly summarize readings, and that might cause students to not read as closely as they might otherwise. What are some of the benefits and also the potential harms that may result from student use of this capability?

Marc: When I first really got into exploring generative AI, really before ChatGPT was launched, there were a lot of developers working in this space, and everyone was playing around with openAI’s API access. And so they’re like, ”Hey, what would you like to build? And people would go on to Twitter, which is now X, and Discord and basically say, “I would like this tool and this tool.” And one of the things that came about from that was a reading assistant tool, which was called Explainpaper. And I think I first played around with this in the fall 2022, and then deployed with students in the spring of 2023. And the whole idea that I had with this and that design was to help students really plow through vast amounts of papers and texts, and so students that have hidden disabilities, or announced disabilities with reading and comprehension, and also students that were working on language acquisition, if you’re working in a second or third language, this type of tool can be really helpful. So I was really excited and I deployed this with my students in my class thinking that this is going to help so many students that have disabilities, that will go through a very challenging text, which is why I set it up as, and it did. The students initially reported to you that this was great. And I met with a lot of my students, and one of them said that she’d had dyslexia her whole life and never wanted to talk about it, because it was so hard and this tool for her was a lifesaver. And so that was great. But then the other part of the class basically said, “Hey, I don’t have to read anything at all ever.” And they don’t have any issues, they were just going to offload the close reading skills. And so I had to take a step back and say, “But wait, that’s not what we want this to actually happen. We want you to use this if you get into a pain point in your reading process, and not completely offload that.” So this really became this kind of a discovery on my part that AI can actually do that, it can generate summaries from vast amounts of texts. There are some really interesting tools that are out there right now: Google’s notebook LM, you can actually upload, I think, 4 million words of your own text to it in 10 different documents, and that will summarize and synthesize that material for you. And like the other tools we played around with the Explainpaper, it can change the summary that it’s generating for the actual document to your own reading level. So you could be reading a graduate level research paper, and you’d like it to be read in an eighth grade reading level, it will change the words and the language of that. So yeah, that could have helpful impacts on learning. It could also lead to a lot of de-skilling of those close reading skills we value so much. So that’s really how this started, was kind of coming up here too, and thinking about “Man, this was such a wonderful tool. But oh my gosh, how is this actually being used? And how has this been marketed to students through social media?”

Rebecca: How do you balance some of these benefits and harms?

Marc: By banging my head against the wall and screaming silently into a jar of screams?

Rebecca: I knew it.

Marc: Yeah, the problem with the jar of screams is every time I open it, some of the screams I put in there before escape before the new ones can come in. That’s a great question. So every single one of these use cases we’re gonna talk about today has benefits but also has this vast sort of terror of being offloading the skills that we would associate with them that are crucial for learning. The most important thing to do at this stage is just to make sure the faculty are aware that this can happen and that this is a use case, that’s the first step. Then the next step is building some friction into the learning process that’s already there. So for reading as an example, something that we do usually is assign close reading through annotation, whether that’s a physical pen and paper, or you could use digital annotation tools like Perusall or Hypothesis to help you go through that, that slows down that process if you’re using AI, and really focuses on learning. So when I say friction, it’s not a bad thing, and point of fact too friction… it’s actually sort of crucial for learning. The one challenge we’re faced with most of these tools is that they’re providing or they’re advertising a friction-free experience for students. And we want to say to them, “Well, you may not want to offload these skills entirely, you want to make sure that you do this carefully.” The main thing too I would think about this is I could never ban this tool even if I wanted to, because you don’t have any control over what students use to read outside of the three hours or so that you’d have in class with students a week. And it would be very beneficial for those students. So we can discuss to look forward to that had all those issues to use it. It’s just basically persuading them to use this in a way that’s helpful to them.

John: It reminds me a little bit of some of the discussions years back on the use of things like Cliff’s Notes for books and so forth, except now it’s sort of like a Cliff’s Notes for anything.

Marc: Indeed, Cliff Notes on demand for anything you want, wherever you want it, however you want it, too. And so how we could do that… what I’d said to my students at the time to kind of get them to be shocked of this is that, “You know, what would your reaction be if I used this to read your essays instead of going through and reading all of it and just giving a nice little generative summary” and one of my students said, “Well, you can’t do that. That’s cheating, you’d be fired.” And I had to explain to them, no one even really knows that this exists yet. There’s no rules. There’s no ethical framework. That’s something we’re going to have to come up with together, both faculty and students talking with each other about this.

Rebecca: It seems like the conversations you were having with students about how to maybe strategically use a tool like this, in this particular way, was an important part of harnessing the learning out of the tool, rather than the quote- unquote cheating aspect of the tool.

Marc: Oh, absolutely. Yeah, I mean, the thing we’ve been seeing with every single generative tool that’s been released too, whether it’s for text generation, or for augmenting reading, or doing some of the other use cases, we’ll talk here today, it does take a lot of time and effort on the part of the instructor to basically say, “Look, this is how this tool should be used to help you in this context in our classroom. How you use this outside of the classroom, that’s gonna be on you. But for our intents and purposes here, too, I would like to advocate that you use this tool this way. And here’s the reasons why.” Now asking every educator to do that is just too much of a lift, right? Because most of our folks are just so burnt out with everything else that they have to do. They’re focused on their discipline-specific concerns. They’re not really even on the radar, the fact that this technology exists, let alone how to actually deal with it. Trying to do part of the series is obviously advocating for people to be aware of it. But the next step is going to be building some resources to show how they can use things like annotation and why that matters. And a very quick way for teachers regardless of discipline to start using in their classes.

Rebecca: Your second post in this series examines the effect of AI tools on student notetaking skills. Can you talk a little bit about what might be lost when students rely on AI tools for notetaking and how it might be beneficial for some students as well?

Marc: Yeah, so a lot of the tools are using assisted speech generation software to actually record lecture like we might be using right now on this podcast and a lot of other people are too, and how they’re being marketed to students is just to sort of lean back, take a nap and to have the AI listen to the lecture for you. And some of the tools out there, I think one of them, it’s called Turbolearn.ai, will also synthesize the material, create flashcards for you, create quizzes for you, too. So you don’t have to do that processing part within your mind, which is the key thing. So, notetaking matters. In fact, it can be an art form. I’m not saying that our students treat notetaking like an art form either too, but there are examples of this that is somewhat of an artistic talent, because you as the listener are not just taking down verbatim what’s being said, you’re making these critical choices, these judgments to record what matters and put it in context of what you think you need to know. And that’s an important part of learning something. One thing that I did too as a student when I was in a community college in Missouri as a freshman, I volunteered as a note taker, and back then we did not have assistive technology. I had a pad of paper for myself for my notes and I had a pad of paper that had larger areas to write in for a student who was functionally blind. So I would do two notes at the same time. One in a font that was my size, one was a larger font that he could read with an assistive magnifying glass from one good eye that he had, it was shocking to me that this is what they did. So the first part of the class is do we have anyone who could help take notes? I was like, “Okay, sure I can.” And that’s how that student had notes for him. Obviously having a system like this in place helps those students so much more than having a volunteer notetaker go through this that’s rushing between one set of notes and another too. And using that in an effective way that’s critical, that is thoughtful about how you’re going to engage with it to, is meaningful for their learning versus just hanging back, sitting down letting the AI listen to you for lecture forty.

John: And another mixed aspect of it is the fact that it does create those flashcards and other things that could be used for some retrieval practice. That aspect, I think, could benefit a lot of students. And not all students maintain a very high level of focus and sometimes miss things. So I think there could be some benefits for everyone, as long as they don’t completely lose his skill. And I think maybe by reminding them of that, that could be useful in the same sort of way you talked about reading. But it’s a lot of things to remind students of. [LAUGHTER]

Marc: That’s lot of things to remind them of, too. And keep in mind, it’s a lot of temptation to offload the skills of learning to something that’s going to supposedly promise you to do that skill for you, or do that time-intensive skill for you too. I would love to have this employed in a giant conference somewhere. In fact, I’d love to go into the hallway of a conference and see all these transcripts come together at once in the overhead almost like you’re waiting for a plane flight at your airport, and you’re just seeing the actual material go through there too. That would be exciting for me too, to see what other people are talking about too… maybe I want to pop into this session and see that as well. So I think there’s tons of legitimate use cases for this. It’s just where’s the sort of boundaries we can put in place with this. And that’s true for almost all of this. I was talking to my wife last night, and I said, “When I was growing up, we had a go kart that a few kids in our neighborhood shared and it had a governor on the engine that made sure that the go kart wouldn’t go past 25 miles per hour, because then you’d basically die because it’s a go kart, it’s not really safe.” None of these tools or these technologies have a governor reducing their ability to impact our lives. And that’s really what we need. The thing that’s shocking about all this is that these tools are being released in the public as a grand experiment. And there’s no real use cases about or best practices about how you’re supposed to use this for yourself in your day-to-day life, let alone in education, in your teaching and learning.

Rebecca: I mean, anytime it feels like you can take a shortcut, it’s really tempting, the idea of turbo learning sounds amazing. I would love to learn really quickly. [LAUGHTER] But the reality is that learning doesn’t always happen quickly. [LAUGHTER] Learning happens from mistakes and learning from those mistakes.

Marc: Absolutely. It happens through learning through errors, it happens through learning through friction in many times. We don’t want to remove that friction completely from that learning process.

John: In your third post in the series, you talk about automated feedback and how that may affect both students and faculty. How does the feedback generated from Ai differ from human feedback and what might be some of the consequences of relying on AI feedback?

Marc: Well, so automated feedback is something that generative AI models, especially large language models, are very good at. They take an input based off of the students writing or assessment, and then the instructor can use a prompt that they craft to kind of guide the actual output of that too. So the system I used in the, I think spring of 2023, maybe it’s the fall of 2023 was MyEssayFeedback designed by Eric Kean. And he’s worked with Anna Mills before in the past too to try to make this as teacher friendly, as teacher centric, as possible, because I would get to design the prompts, my students would then be able to get feedback from it. And I use this in conjunction with asynchronous peer review, because it’s an online class. So they got some human feedback, and they got some AI feedback. The thing that was kind of shocking to me was that the students really trusted the AI feedback because it’s very authoritative. It was very quick, and they liked that a lot. And so I did kind of get into the situation where I wanted to talk with them a little bit more critically about that, because some of the things I was seeing behind the scenes is that a lot of the students kept on cueing the system over and over again, they’d get one round of feedback from the tool, they would try to go back and using air quotes right now so your audience can see this “fix” their essay. And my whole point is their writing is not broken. It doesn’t need to be fixed. And generative AI is always going to come up with something for you to work on in your essay. And one student I think went back seven or eight times saying “Is it right now? Is it perfect?” And the AI would always say something new. And she got very frustrated. [LAUGHTER] And I said “I know you’re frustrated, because that’s how the AI is. It’s not smart, even though it sounds authoritative, even though it’s giving you some advice that is useful to you. It doesn’t know you as a writer, it doesn’t understand what you’re actually doing with this piece.” So that crucial piece of AI literacy, knowing that what the limitations are too, is a big one. I think also when you start thinking about how these systems are being sold, in terms of agentic AI, we’re not there yet. None of these systems are fully agentic. That involves both strategic reasoning and long-term planning. When you can see that being put in place with students and their feedback, that can become very, very scary in terms of our labor for faculty to understand that, because there are some examples of some quirky schools, I think it’s the Health Academy in Austin’s one of them that have adopted AI to both teach and provide feedback for students. And I know there’s some other examples too, that talk about the AI feedback being better than human feedback in terms of accuracy. And that is something that we are going to have to contend with. But when I provide feedback for my students, I’m not doing it from an aggregate point of view, I’m not doing it to try to get to the baseline, I want to see my student as a human being and understand who that writer is, and what that means to them. That’s not saying that you can’t have a space for generative feedback, you just want to make sure you do so carefully and engage with it in a way that’s helpful for the students.

John: And might that interfere with student’s development of their own voice in their discipline?

Marc: I think so. And I think the question we don’t have an answer to yet is what happens when our students stop writing for each other or for us and start writing for a bot? What happens when they start writing for a robot? That’s probably going to change their voice and also maybe even some of their ideas and their outlook on the world too, in ways that I’m not all that comfortable with.

Rebecca: It does seem like there’s real benefits to having that kind of feedback, especially for more functional things like grammar and spelling and consistency and that kind of thing. But when you lose your voice, or you lose the fresh ways of saying things or seeing things in the world. [LAUGHTER] you lose the humanity of the world, [LAUGHTER] like it just starts to dissipate. And to me, that’s terrifying.

Marc: It’s terrifying to me too, to say the least. And I think that’s where we go back into trying to find, where’s the line here? Where do we want to draw it? And no one’s doing it for us. We’re having to come up with this largely on our own in real time.

Rebecca: So, speaking of terrifying [LAUGHTER] and lines, you note about how large language models are developing into large multimodal models that simulate voice, vision, expression, and emotion. Yikes. How might these changes affect learning, we’ve already started digging into that.

Marc: Yeah, so this is really about both Google’s demo, which is I think called Project Astra and also openAI’s demo, which is GPT4 omni. Half of the GPT4 omni model is now live for users, you can use the old version of the large language model too for resources, but the other half is live streaming audio and video. And the demo used a voice called Sky that a few people, including Scarlett Johansson, said “that sounds an awful lot like me.” And even the creator of openAI, Sam Altman, basically said that they were trying to go for that 2013 film Her where she started as the chatbot to Joaquin Phoenix. And basically, this is just the craziest thing I can ever think of. If openAI goes through with the promise of this, it will be freely available and rate limited for all users. And you can program the voice to be anything you want, whenever you want. So yes, it’s gonna be gross and creepy, there’s probably going to be people that want to date Sky or whoever it is. But even worse than that, there will probably be people who want to program this to be a political bot. And they only want to learn from a liberal or conservative voice, if they only want a voice that is of their values and their understanding of the world. If they don’t like having a female teacher, maybe they only want a male voice talking to them. Those are some really, really negative downstream effects of this that go back into how siloed we are right now with technology anyway, that you can now basically create your own learning experience or your own experience, and filter the entire world through it. We have no idea what that’s going to do to student learning. Sal Khan thinks that this is going to be a revolution, he wrote about this in Brave New Words. I think that this is going to be the opposite of that. I think it’s going to be more chaotic. I think it’s also going to become, for us as teachers, very difficult to try to police in our classes, because at my understanding of this, this is a gigantic privacy issue. If your students just come up and you’re having a small group discussion or anything else that’s going on too, and one of them activates this new multimodal feature in GPT4 omni and there are voices streaming, they’re talking to the Chatbot and everything else, anything that goes into that is probably going to be part of its training data in some way, shape, or form. Even Google’s demo of this using Project Astra, part of the demo was actually having someone walk around a room in London, they had stopped on a computer screen that was not the actual person’s computer screen and it had some code running for encryption and it read the encryption out loud. It said what it was. So there’s some big time issues that are coming up here too. And it’s all happening in real time. We don’t even have a chance to basically say, “Hey, I don’t really want this,” versus “Oh, this has now been updated. I now have to contend with this live in my own life and in my classes.”

John: Going back to that issue of friction that you mentioned before, Robert Bjork and others have done a lot of work on the issue of desirable difficulties. And it seems like many of these new AI tools that are being marketed to students are designed to eliminate those desirable difficulties. What sort of impacts might that have in terms of student learning and long-term recall of concepts.

Marc: I love desirable difficulties too, and I think that’s a wonderful framing mechanism outside of AI to talk about this too, and why learning really matters. I think the downstream consequences that if this is widely adopted by students, which I think a lot of tech developers want this to happen, and we don’t see this sort of sporadic usage. which we’re seeing right now… to be clear to your audience, not every student is adopting this, not everyone’s using this, most of them are really not aware of it. But if we do see this widespread adoption of this, too, it is going to have a dramatic impact on the skills we associate with reading, the skills that we associate with creating model citizens who are critical thinkers and ready to go into our role to actually participate in them. If we really do get to the situation where they use these tools to offload learning, we’re kind of setting up our students for being uncritical thinkers. And I don’t think that’s a good idea.

Rebecca: Blah. [LAUGHTER] Can you transcribe that, John? [LAUGHTER]

John: I will. I had to do a couple of those. [LAUGHTER]

Marc: Well, blah is always a great version of that. Yeah. [LAUGHTER]

Rebecca: I only have sound effects.

John: One of the transcripts mentioned “horrified sound” as the transcript.[LAUGHTER]

Rebecca: I think that’s basically my entire life. These are the seeds of nightmares, all of them… seeds of giant nightmares.

Marc: Well, I think the thing too, that’s so weird about this is that, yes, and this is kind of getting into the dystopia version of it, but there’s clearly good use cases for these tools, if you can put some limitations on it. And if the developers would just sort of pause and think not just as someone wanting to make money, but as someone who would use this tool to actually learn or be useful to their lives, what areas do they want to design to actually preserve that sort of human judgment, that human sort of friction in learning is going to be meaningful for that going forward?

Rebecca: Yeah, guardrails and ethics would be great.

Marc: Absolutely.

Rebecca: So a number of these tools are also designed to facilitate research. What’s the harm? What harm might there be when we rely on AI research tools more extensively, and get rid of that human judgment piece?

Marc: Yeah, I think one of the tools I used initially was Elicit and Elicit’s probably the most impressive research tool that’s currently available. It is expensive to use, so it’s hard to sort of like practice using it now. It was free initially. Consensus AI, I think is the best ChatGPT plugin that you can use through the custom GPT store. But what Elicit does is it goes through hundreds, if not 1000s, of research papers, and it automates the process of reading those papers for you, synthesizing that material, and giving you an sort of aggregate understanding of the state of knowledge, not just within your research question, but perhaps even in your field of research you’re trying to acquire. So you’re basically offloading the process of research, which for a researcher to do that, takes hundreds upon hundreds of hours of dedicated work, and you’re trusting an algorithm that you can’t audit, you can’t really ask how it came up with its response. So yes, it’s a wonderful tool, when it works and when it gives you an accurate response. Sometimes the responses are not accurate in the least. And if you haven’t read the material too, it’s very difficult to sort of pick up on where the machine is making an error. So yeah, there’s a lot of issues if we just uncritically adopt using this tool, versus if you sort of put some ground rules and ethics about how to use this, to support your research, to support your learning as well. And I think that’s what we want to try to strive for with all of these. And research is just one level of that.

Rebecca: We all have our own individual assumptions that we make when we do things, many of which we’re not aware of. But when we’re relying on tools like this, there’s many more layers of assumptions that we might not be aware of that are built into the software or into the tools or in the ways that it’s doing its analysis or synthesis that I think seems particularly concerning to me.

Marc: Yes, the bias, the sort of hidden biases that we’re not even aware of. And then the developers I don’t think are aware of either, too, is another layer that we can go into and think about this. I say that layer, because this really is like an onion, you peel back the layer, there’s another layer there, another layer, another layer, you’re just trying to get to the point where it’s not so rotten anymore. And it’s very difficult to do because the way that this has been shaped to do is to just accelerate those human tasks as quickly as you can to reduce as much friction as possible, so that you can just sit back and get a response as quickly as you can from this. And in a lot of ways the marketing of this basically describes this as almost like magic. Well, it’s not magic, it’s just prediction and using massive amounts of compute to get you to that point as well, but there are some serious consequences, I think, to our learning if we just uncritically adopt that.

John: Going back a bit, though, to early in my career, I remember the days of card catalogs and indexes where you had to read through a lot of material to find references. And then finding more recent work was almost impossible unless you happen to know of colleagues doing this work at some other institution, or you had access to the working papers of other institutions because of connections. The fact that we have electronic access to these files, and you don’t have to wait a few weeks for one to be mailed to you, or go through interlibrary loan. And that we can do searches and get indexes or get abstracts, at least for these articles, takes us a long way forward. And one other thing is that I do subscribe to Google Alerts in some of my popular papers. And then I occasionally, maybe once every month or so when I see some new ones, I’ll just look at the article and about half the time the person who cites the article gets it wrong, they actually refer to it in a context that’s not entirely relevant. I think in some ways, maybe relying on an AI tool that generates some summaries of the articles before people add them to their bibliography or footnotes, might actually, in some cases, improve the work. Going back again to the early days, one of the things I enjoyed most when I was up there in the periodical sections of the library were the articles around the ones that I was looking for, they’d often lead to some interesting ideas. And that doesn’t come up as much now when you’re using an online search tool, but as you’ve noted all along, we have both benefits and costs to all this. And in this issue, I’m kinda thinking some of the benefits might be worth some of the costs, as long as people follow through and actually read the articles that seem relevant.

Marc: I think that’s the key point too. So long as this leads you to where you want to go. That’s just like what Wikipedia basically is, that’s a great starting point for your research, it just leads you back to the primary sources to actually go in there and read to do it. The challenge that I think we see, and this is what it goes back down to where we go back that onion sort of analogy, is that a lot of the tools that are out there now …I think one of them is called ProDream AI or something like this… will not only find the sources for you, but then it will draft the lit review for you as well. So you don’t have to go through that process of actually reading it. And obviously, that’s where we want to pause and say this isn’t a good idea. But I agree with you completely. John, we are in a digital age, we have been for over 25 years now too. And in fact, when I students is: “This was a terrible experience because I can’t navigate this thing. This is just so horrible for me to do.” And yet every time I’ve done this with the AI research for my students, the interface design is much more easy for them to actually establish and look at sources and go through this and think about it, and part is because the algorithm is now using some of those techniques to actually narrow down their sources too and help them identify them as well. So yeah, there’s definitely benefits to it. It’s not all black and white, for sure.

Rebecca: There’s a lot of gray. [LAUGHTER] I think one of the things that you’re hinting at too is this difference between experts using a tool and novices or someone who’s learning a set of skills. And the way that these tools are designed, an expert is going to be able to use a tool and have a judgment call about whether or not what’s provided is accurate, helpful, relevant, etc. Whereas a novice doesn’t know what they don’t know. And so it becomes really challenging for them to have the information literacy skills that may be necessary to negotiate whether or not this is a path to follow or not. For me, that’s one of the biggest differences when we’re talking about using these tools in a learning context versus using these tools in a professional context are ways to save time to get to the point or get to an end result more swiftly.

Marc: Oh, absolutely. I think that thinking about the audience who’s using it too: a first-year true freshman student, using a tool like this versus a third-year PhD student working on their thesis is a totally different audience, totally different use case. For the most part, the PhD student hopefully has that literacy needed to effectively use these tools already, they might still need some guidance, might need some guardrails and some ethical framing for this too, but it’s a very different situation from that freshman student. I think that’s why most faculty aren’t thinking about how they’re using these tools, because they already have many of those skills already solidified. They don’t need to have a refresher course necessarily on research because they’ve done this now for a large part of their career. For their perspective, adopting these tools is not going to necessarily de-skill them, it might just be necessarily a timesaver in this case.

Rebecca: And what skills we’re offloading to a tool. Some things are just repetitive tasks that take a long time that a tool is great to solve. Just a kind of a waste of time versus really like critical thinking or kind of creative aspects of maybe some of the work we do.

Marc: The tool I want, and I think this exists, I just haven’t found it yet is when I’m trying to write a post and instead of trying to search for the URL to go into the actual title that automatically just finds the URL for me to click on it. I’ll review it for a second, because it takes me so much time finding the URL for the page when I’m doing either a newsletter or I’ve tried to update a website, that would be amazing. Those are some of the things that we could use really easily to cut down on those repetitive tasks, for sure.

John: In your six post in this series, you talk a little bit about issues of ethics. And one thing that I think many students have noted is that many faculty have extremely different policies in terms of when AI is allowed, if it’s allowed, and under what conditions it’s allowed, which creates a lot of uncertainty, and faculty aren’t always very good at conveying that information to students. What should we be doing to help create perhaps a more transparent environment for our students?

Marc: Well, I think transparency is the key word there. We want to, if we’re using these tools for instructional design, be transparent about what we’re using this to, just to model that behavior for our students. So if I develop a lesson plan or use a slide deck that has generated images, I want to clearly identify what part of AI was in that actual creation and talk about why that matters in these situations. What concerns me is that these tools are being turned on left and right for faculty without any sort of guides or best practices about that. I actually asked for Blackboard to have a feature built in with a new AI assistant, so it could identify what was AI generated with a click of a button. There’s no reason why you can’t build something that tracks what was generated by AI within the learning management system. And the response that I’ve gotten to is: “Who basically cares about that?” Well, I kind of care about that, and I care about this for the effects we’re trying to do for our students as well. But yeah, I think adopting a sort of stance of transparency as a clear expectation, both for our own behavior and our students behavior is going to be more meaningful than turning to sort of an opaque AI detector that’s only going to give you a percentage about if this is aggregated content or human content or completely misses the entire situation and misidentifies a human being as AI or vice versa. And that’s something I think we want to focus on as being that human in the loop situation here too. And really not offloading ethics in this casein just trying to teach it. It is hard to do that when the technology is changing rapidly before your very eyes, though. And that’s what this has felt like now for the last two years, I think.

Rebecca: You’re really concerned when faculty lean on an AI detection tool as the only way of identifying something that might be AI generated or an academic integrity violation of some sort. Can you talk a little bit about the effectiveness of these tools, and when they might be useful and when they might not be useful?

Marc: Yeah, to me, they’re not very reliable in an academic context, there’s far too many false positives. And more importantly, too, the faculty that employ them, for the most part, aren’t really trained to actually use them. So some universities have invested in academic misconduct officers, academic honesty officers, or whatever you call them, for offices of academic misconduct, where they actually have people who are trained to both use these tools and provide this to faculty. I might be a so-called expert at AI, again, I’m gonna use air quotes here too, because I’m self taught like everyone else is. But I don’t think I would be comfortable in an academic based conduct investigation, trying to use these tools, which I barely understand how they work, trying to come up with a case for students to do so. The few areas that I’ve looked at that have engaged AI detection, do so as part of a process. And that process is just one part of the AI detector, they have independent advocates usually coming in talking with the students and talking with the faculty member, they don’t go to taking students up on charges at the first step, they often try to look at a restorative process to see if that’s possible. So if the first instance of a student using this technology, they would sit down, and they would be like a third party between the instructor and the student, and talk about if something could be repaired within the relationship. And if the student would acknowledge that an ethical breach actually happened here, not rule breaking, but an ethical breach that has damaged this relationship. And can that relationship be basically restored in some way. So to me, that’s the gold standard of trying to do this, that takes a whole bunch of resources to set up, lots of training, lots of time, versus let’s buy an AI detector for our entire university, turn this on and here’s a little one-page guide about how to use it. And that, to me, has set up this recipe for just chaos in the world too. And it doesn’t matter what detector you’re using. They all have their own issues. And none of them are going to ever give you a complete picture of what’s going on with that student. And I think the big challenge we’re seeing too is that we’re moving well beyond AI detection into some pretty intense surveillance. We’ve got some companies going to stylometry and going through keystroke logging, tracking what was copied and pasted into a document, when it was copied and pasted to. And these are all interesting novel techniques to try to figure out what was written and who wrote it, but they also have some downstream consequences, especially if they don’t involve training. I can imagine certain faculty using that time stamping technique to penalize students by not spending enough time on their writing, whether there is AI in it or not, they’re looking at: “you only spent two hours on this essay that was assigned over two weeks, that’s not showing me all you’ve learned, other students spent 5, 6, 7,12, 14 hours on this. So I think we have to be really careful about what comes online these next few years, and really approach it critically, just like we are asking our students to, so that we don’t look for a solution for this problem that’s based on technology.

John: One of the things you discuss in this essay, though, is the use of digital watermarking, such as the work that Google has been doing with synthID. Could you talk a little bit about how that works, and what your thoughts are about this.

Marc: So watermarking has been sort of on the perpetual horizon in AI for a long time. I think Scott Aaronson, he teaches at the University of Texas, he has been working with open AI for the last two or three years, he has really been very vocal about his own research into watermarking. And supposedly, he has a watermarking system at OpenAI working in the background, they just have not deployed it in public. Google’s synthID is not just for text, it’s for images, it’s for audio, it’s for video. And it’s really designed for what our world is going to very soon look like when you can have an AI that makes the President say anything, do anything, and deal with this vast amounts of misinformation and disinformation, too. And so what synthID is is their actual watermarking technique, and watermarking starts at the source of the generation. So their model was Gemini. And when watermarking comes online, it uses cryptography to put a code into the actual generation, whether that’s a picture, a video, music, or text that can only be deciphered from a key that they actually have. And so watermarking is this really interesting technique that it can be used to try to identify what was made by machine versus a human being. The challenge is, the last time I checked, there’s almost 70 different models on the market now that use multimodal AI or large language models. And those are the only ones I’ve been tracking, I’m sure there’s probably hundreds of others that are small that people have been developing, Google’s synthID model is specific to Google’s products, all the other watermarking schemes will be absolutely specific to OpenAI or Microsoft or Anthropic or any other companies. So it’s going to be the situation where you’re going to use a tool, then you have to rely on the tool to give you a classification if this is accurate or not. And from what I’ve also read, it’s pretty easy to break, because you can feed it into an opposing system’s AI or an open model. And it will simply rewrite it, removing the actual code in that process. So I don’t think watermarking is going to be a long-term solution, I do think it’s a good first step towards something that we can actually do. But it’s just a little bit too chaotic right now in the space. And we would need some massive sort of multinational treaties with different countries who don’t like to talk with us to try to get a sort of unilateral watermarking scheme in place that everyone will agree upon. And then we’d all have to cross our fingers that that key would never be released to the public. Because if that ever happened, that’s when the whole sort of house of cards falls apart.

Rebecca: So that’s kind of a fantasy.

Marc: …kind of a fantasy, but part of this stuff, I think, is marketing based. So like Google wants their products to be both safe and secure. You can’t have that safety and security unless you have some sort of system between there. And that’s what synthID is. I think that it can possibly work for audio, for video, and even for images. I think text is a lot more fungible than anything else, because it’s very easy to start copying and pasting things out there too. It’s also easy to write as yourself as a human being into a document. And that becomes very difficult to sort of gauge what was human versus AI using a watermarking type of program like this.

Rebecca: The final post in your series addresses the use of generative AI tools to design instructional content and activities. Instructors often find the use of AI tools to be very useful for these purposes, even if they ban it for their students. What concerns do you have about relying on AI tools in this context?

Marc: My concern there: “AI for me, not for you. It makes perfect sense to me going forward.” Yeah, obviously we go back into this phase of trying to model ethical behavior using the tools too and understanding why this matters. If you’re going to use a tool to grade or design rubrics, you want to be open about it. You want to be attributing what you use this tool for too, because your students are going to be looking at you and seeing “Well, how are you using this in your job? How am I going to be using this in my job when I graduate from here too?” That’s the actual grounding framework we can do for this for our students and for ourselves. If we can think about that and do that, then we don’t have to necessarily rely on technology as being the sole solution for this, we can start talking about “this is the ethical behavior I’m modeling for you too, this is the ethical behavior I expect from you too. Let’s work together and think about what that means.” Now, that’s not always going to be the solution for this situation, some students are going to listen to that, other students are going to smile at you and go back and happily generate away and try to get past it. But the fact is, we do have that agency in our part too. And that is something I think we should be leaning into right now. Because the connections we’re developing with our students too are, as of this time, still human-to-human base, for the most part. I want to value that and use that to try to persuade them on an ethical pathway.

Rebecca: Modeling our use of technology leads to so many different interesting conversations with students. I know that when I’ve talked about using assistive technology in my classes, having something to read to you if you’re having trouble focusing or using some of these tech tools to solve barriers that you’re facing in getting your work done. And sharing the ways that you use tools to do the same can be really helpful in leading to student success. So I can see how doing the same thing when it’s an AI product is relevant. I know that I used AI to generate a bunch of little case studies for one of my classes and I just told the students that that’s what I did… fed it in a prompt, and I made some tweaks to it, but this is where it came from. And they found it really interesting, and we ended up having a really interesting conversation about when it might be most relevant to use particular tools and when maybe it’s not as wise to use a particular tool, because it isn’t actually helping you in any kind of way. Or it’s defeating the learning, or it’s not really creating a good product in the end.

Marc: That’s a wonderful use case. I mean, sitting down there talking with them and saying how I use this, why use this, let’s get into discussion about this, maybe even a debate about that, too, is part of the learning process. And I’m glad you focused on the fact that about the assistive technologies, I want my students to use this technology if they need to, they don’t need to announce that they have a disability. We need to really be focusing on this fact, for education beyond. At our university, they have to go through a very formalized process to be recognized by the Office of Student Disabilities. It’s very expensive, it’s time consuming, that is out of reach for the vast majority of students, even if they felt comfortable going out there and advocating for themselves that way or if they had parents or other resources to do that. I want to design my classes so that students are aware that these tools exist, that they can use them and that they can be able to trust them to hopefully be able to use this in a way that is effective to their learning too and to trust them for that. That’s what I want. Now if that’s going to happen is another case, indeed. But that’s going to take time. The one thing I will say too, and I think that something that popped up here at a recent story that I read is that professors were moving from a point of despair to anguish with this technology, and I want us to avoid that more than anything else. Because that’s not the sort of stance we need to be taking for ourselves when we deal with this technology with our students. We can navigate this, it’s just going to take a lot of time and a lot of energy. And I hope administrations of various institutions are listening to that too, that they really need to focus on the training aspect of this technology, both for students and for actual teachers. This isn’t just something you flip a switch and turn on and say: “You guys now have AI, go learn how to use it…” that has been a recipe for disaster for it.

Rebecca: It’s definitely a complex topic, because there’s so much hope for equity in some of these tools, especially for students with disabilities. But then there’s also the really scary parts too. [LAUGHTER] So finding that balance, and making sure that both enter conversations when we’re having conversations about AI, I think, is really important. And I appreciate that today we’ve done that, that we’ve talked about some of the scary aspects, but also there’s some real benefits to having these tools available to our students and incorporating them and really having deep and meaningful conversations about them.

Marc: Absolutely. I think that one of the most powerful things I’ve done from the AI Institute is when you can get a skeptic and an early AI adopter at the same table together talking about these things back and forth. You really do see how people come out of their sort of silos and their positions and they can kind of come together and say “Yes, this is an actual use case or two. This is actually meaningful. This is good. How do I make sure that I can put some boundaries on this for my own students and their learning?”

John: So, we always end with a question which is so much on everyone’s mind concerning AI, and that is: “what’s next?”

Marc:Well, what is next indeed? So I think we’re all holding our breath to see if OpenAI is going to fulfill its promise and asking if they’re going to turn on this new multimodal system that lets you talk with it, lets it see you, because they have not done so yet. So we have a little bit of time. But that is going to be on everyone’s mind this fall if they do so. Because having an AI that can listen to you, talk with you, and have a voice that you get to program it, is going to be a new set of challenges that we have not really come up with yet.

John: Well, thank you. This has been fascinating, and your series is wonderful. And I hope that all faculty think about these issues, because a lot of people are focusing on a very narrow range of issues and AI is going to affect many aspects of how we work in higher ed.

Marc: Thank you, John. Thank you, Rebecca. This has been great too. And hopefully I’ll be putting some more resources into that series [LAUGHTER] when I have a chance to do so here.

John: And we will include a link to your substack in the show notes because you’ve got a lot of good information coming out there regularly.

Marc: Thank you.

Rebecca: Well, thanks for joining us. We hope to talk to you again soon.

Marc: I appreciate it. Thank you guys.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

352. Enhancing Inclusive Instruction

We often don’t have the opportunity to hear directly from students about inclusive teaching practices. In this episode, Tracie Addy, Derek Dube, and Khadijah A. Mitchell, the authors of Enhancing Inclusive Instruction, join us to explore how student perceptions of inclusive teaching practices align with the growing consensus on what constitutes inclusive teaching.

After serving as the Associate Dean of Teaching and Learning at Lafayette College, Tracie will be transitioning to a new role this summer as the Director of the Institute for Teaching, Learning, and Inclusive Pedagogy at Rutgers University – New Brunswick. Derek Dube is an Associate Professor of Biology and the Director of the First-Year Seminar Program at the University of St. Joseph in Connecticut. Khadijah A. Mitchell is an Assistant Professor in the Cancer Prevention and Control Program at the Fox Chase Cancer Center in the Temple University Health System and Affiliated Faculty in the Department of Epidemiology and Biostatistics at the Temple University College of Public Health.

Show Notes

Transcript

John: We often don’t have the opportunity to hear directly from students about inclusive teaching practices. In this episode we explore how student perceptions of inclusive teaching practices align with the growing consensus on what constitutes inclusive teaching.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane , an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are: Tracie Addy, Derek Dube, and Khadijah A. Mitchell. After serving as the Associate Dean of Teaching and Learning at Lafayette College, Tracie will be transitioning to a new role this summer as the director of the Institute for Teaching, Learning, and Inclusive Pedagogy at Rutgers University – New Brunswick. Derek Dube is an Associate Professor of Biology and the Director of the First-Year Seminar Program at the University of St. Joseph in Connecticut. Khadijah A. Mitchell is an Assistant Professor in the Cancer Prevention and Control Program at the Fox Chase Cancer Center in the Temple University Health System and Affiliated Faculty in the Department of Epidemiology and Biostatistics at the Temple University College of Public Health. Welcome back Tracie, Derek, and Khadijah.

Derek: Thank you so much.

Tracie: Thank you.

Khadijah: Thank you so much.

Rebecca: Today’s teas are:… Tracie, are you drinking tea today?

Tracie: I am drinking tea. And I love that I’m drinking it from my Tea for Teaching mug that John gave me at the POD conference in November. And the tea that I’m drinking is just a simple kind of peppermint tea.

Rebecca: Sounds wonderful, and in the best mug possible. [LAUGHTER] How about you, Derek?

Derek: I’m just drinking water. But I would also like to shout out to my mug, which is actually a mug that I recently received from an instructor in my first-year seminar program here where the second semester of our first-year seminar program is a service-based learning course. And this mural that you see on the mug, and also behind me today, is one that their class created as a means to, as it says, inspire and engage and unite the globe around topics of digital and media literacy and things like that. So shout out to the mug, more so than the beverage within the mug.

John: It’s a very impressive mug and image behind you.

Rebecca: And something good to drink to, for sure. [LAUGHTER] How about you, Khadija?

Khadijah: So, I am drinking organic spearmint tea, and my mug has my name on it and in the spirit of inclusivity, this is probably the first thing I’ve ever had with my name on it. [LAUGHTER] When I go on vacation, I never get a keychain. So I’m very excited to drink the spearmint tea out of this mug.

Rebecca: I love a special mug. John and I don’t really have special mug.

John: Well, I do…

Rebecca: Oh you, do. Right.

John:…from the SUNY Oswego School of Ed, because we moved to a new recording room that they’ve loaned to us after we lost our other one. And along with the room, we got a tea mug, a nice insulated one.

Rebecca: And what kind of tea are you drinking?

John: In line with what we’ve just heard, I am drinking a peppermint spearmint blend today.

Rebecca: I missed the mint protocol today, unfortunately.

John: You didn’t check the calendar?

Rebecca: I didn’t. I’m sorry. [LAUGHTER] But I have a nice Jasmine black tea this morning, which feels nice and springy.

John: We’ve invited you here today to discuss your new book: Enhancing Inclusive Instruction: Student Perspectives and Practical Approaches for Advancing Equity in Higher Education. Your previous book, What Inclusive Instructors Do, was the basis of a very well received faculty reading group that we ran at SUNY Oswego, in collaboration with Jessamyn Neuhaus at SUNY Plattsburgh. So this book is a superb follow up. Could you tell us a little bit about the origin of this new book and how it connects to your earlier work?

Tracie: Absolutely, John, pleasure to do so. So when we think about inclusive teaching, and when I think about it, I would say in general, we know that instructors started at different points in their journeys. And when we wrote What Inclusive Instructors Do, we really knew that it will be really helpful just to have the voices of instructors who do that work, and their thinking about how they think about it, their mindsets, their values, etc. But also kind of distill it into some principles and practices that were very practical, because we know that that speaks well to instructors, but also that would allow them to reflect on their teaching. So after What Inclusive Instructors Do, we knew that instructors aren’t the only voice, right? [LAUGHTER] So we also knew that how important it was to really get students involved and students seeing the benefits of inclusive instruction, how did they experience these spaces in terms of their sense of belonging, welcoming environment, the feelings that it’s equitable or fair as well. So that kind of elicited our next step to really do a study on students and to find their perspectives. And a couple of things that we were able to do with this study, which I think is really helpful and useful for anybody, whether they’ve read What Inclusive Instructors Do or not, is we were also able to identify and ask students aspects according to what are the pitfalls? What have you seen not be inclusive? So that emerges kind of more strongly also in this book than the first book, for example. So we were able to really capture their experiences and these are students from a variety of institution types. They’re coming from a variety of classes they’ve been taking, various identity backgrounds, etc. So it really feels like now we’re talking about students like students [LAUGHTER], it’s their voice in this book. So Enhancing Inclusive Instruction, basically is just that, so that we can have a more comprehensive picture understanding of an inclusive classroom. What does it mean to students? What does it look like for instructors? And the beauty of it, and this is something that I’ve been speaking about a lot is, how are these alignments? What are the alignments between these two perspectives? What are the differences between these two perspectives? And how can we use this wider body of knowledge by having these viewpoints to really inform our practices in what we do in the classroom. So in general, both books, you can read both, you could read one, either/or, but they really were meant to really support inclusive instruction, no matter where an instructor is in your journey.

Rebecca: As a user experience designer and interaction designer, I really love the very student-centered or audience-centered nature of this book, and really bringing those voices to the forefront. Could you tell us a little bit about how you found the student participants to participate in your project.

Derek: So much like we set out with the goal of hoping that our teaching is as inclusive as possible, we also wanted that to be the case for our study, and our recruitment in the voices that we were hearing back from. So we made extensive efforts to have as broad a population be able to give us feedback, to participate in our surveys and our interviews. And that’s an important part, there were opportunities for our student respondents to reply via survey or via a virtual interview where they could potentially even go further and have deeper conversations around some of the topics if they wanted as well. So in order to recruit, after many collaborative meetings between myself and Tracie and Khadija, we came up with a broad spending plan to recruit students through using connections that we had, various educational listservs like POD, social media. We directly communicated with various institutions across the country, their teaching and learning center specifically, but also faculty that we knew that were invested in this type of work. And this included institutions at the community college level and research intensive institutions, liberal arts, HBCUs, colleges for students who learn differently, and basically any type of college you can think of. We also did lean on and take advantage of an opportunity to leverage technology through cloud research, to expand our reach as well. And through all of this, we did end up with well over 300 responses from various students when looked at geographically or institutionally that were very diverse. And we felt good with that as a basis for which to move forward.

John: One of the things we really appreciated when we were running the reading group was the reflection questions you had embedded in what inclusive instructors do. And this book also contains reflection questions spread throughout each of the chapters. While most teaching centers do encourage faculty to engage in reflective practice, it’s nice to see that embedded in the books that they’re reading. But might this be even more effective when faculty are reflecting on the student voices that make up so much of the focus of this book?

Khadijah: Right, so one of the things that stood out to us, when we did survey the inclusive instructors, there was a particular quote saying that inclusive teaching is a perpetual work in progress, informed by research, dialogue, and reflection. And when we think about that, in the context of incorporating student voice, it’s a perfect example of this. So when we think about research, actually collecting data from our students thoughout our courses, and I mean that as a community of inclusive instructors, and really fostering dialogue, and to be honest, what we found from what students say it is, that dialogue means we have to be willing to share and discuss and actually negotiate the space in our classrooms. And when we think about reflection, thinking that there’s no one set of practices for every inclusive instructor that we can act, and we’re not superheroes. So you can’t do all things, but you can do what you can to be responsive to your learners’ needs at the same time. So with that being said, when we came up with these reflection questions, and actually thinking about one of those quotes from the instructor, we saw that there were some mirroring with the first book in terms of course design, and creating a welcoming environment and conducting the course. But really, we saw that there were these 13 key themes, and that, although sometimes they reflected what we saw in the first book, we actually saw new themes emerge. So we really wanted to write reflection questions that captured that, that would help you delve deeper in thinking about the student perspective. And I think to echo something that Tracie mentioned, we really thought about potential pitfalls with these questions that we wouldn’t have thought if we didn’t speak with the students from their perspective to basically avoid certain types of things that may not have been the best for course design or maybe didn’t make everyone feel welcome, or when conducting the course. So that was very encouraging, the way that we could tailor these reflection questions based on what the students said.

Rebecca: And I love that so much. One of the tools that you developed and described in the book is the Protocol for Advancing Inclusive Teaching Efforts, or PAITE. Can you describe this instrument for our listeners?

Tracie: Sure. So the Protocol for Advancing Inclusive Teaching Efforts is a classroom observation protocol that enables instructors, in a formative way, low stakes, to obtain feedback on 15 specific instructional behaviors that are considered inclusive, equitable types of behaviors. And this protocol actually emerged from my work. And it’s a lot of what I’ve been doing in my own work is really thinking about tools that can support our instructors in building more inclusive classrooms. And so this PAITE actually originated from that. And we co-created it with students. So the student partners that work on a lot of the inclusivity efforts that I’ve done at my institution, basically supported this work and helped us measure reliability, et cetera. They were like, you know, the researchers, they helped us think about and devise it. So this particular protocol, I think, has been pretty impactful. We’ve done hundreds of observations with it at the institution that I was previously at. And thinking about all the debriefings, and all the feedback, it really supported instructors in thinking about their inclusive teaching practices. In fact, I like to share some of the responses that I’ve seen or heard them have around it. Some of them and even some of the student partners will actually, in future classes, they’ll say, “Am I doing this?” “Am I doing that?” They’ll actually reflect on the codes that are embedded within this protocol and continue to think about their practice. And that’s actually a very positive outcome of this. If they’re actually putting it in their awareness, awareness is critical for knowing are we actually doing these things that we say or hope we are doing in the classroom, and that has actually helped them really think about their teaching differently. And then some of the students said that actually, when they go to their own classes, they’ll start to think about the codes, because they’re so immersed in actually doing a protocol or conducting it with their partners as well. So that has been really great to see. And so we included it in the actual book, it has like a separate methods kind of paper, or how we developed it. But in the book, it really talks about how it can be used, and how it aligns a lot of the behaviors, and I’m actually doing a lot of this interesting mapping, align with what the perspectives of the students are, and also our instructors in What Inclusive Instructors Do. So it provides a really beautiful way to think about your teaching and monitor your approaches, reflect on your teaching, by actually having that data from your actual classroom. And I think it’s just really been helpful. We’ve embedded it in a variety of different academies and institutes, and different programming at my previous institution.

John: So you mentioned the themes that have arisen from the student interviews. Could you talk a bit about those themes?

Khadijah: Sure. So we saw 13 themes. And interestingly, seven of them centered on inclusive course design. So I’ll elaborate on the two that were the most prominent. We were really interested and surprised to see that maximizing student engagement from the first assignment and throughout the course. So the student has highlighted the importance of that first assignment, and several of them, they really believed that it helped them have a high degree of engagement. But also it was important that it required little content mastery, and built upon their existing knowledge. And there was a lot of positive feedback from successfully completing that first assignment. Students referred to short-term as well as long-term positive emotions from that first assignment. So that was really interesting. And group work… we often think about group work in higher education as a seminal active learning strategy, different learning teamwork, and things like that. So a lot of the students also echoed that, that it maximized the engagement. But what we saw them speak about was the pitfalls from their perspective with the group work, but in particular, those students that were neurodivergent and had disabilities. And so some students along with autism spectrum disorder, and also attention deficit hyperactivity disorder mentioned that, in that particular context, they didn’t feel most engaged with that. Student choice was very prominent. We talked a lot about: flexible deadlines were really important, individual student preferences on assignments and exams and accessibility needs and I think many of us are going to talk a lot about mental health and student well being. So that was very clear there. So those two were some of the key things, but also, including course policies that respectfully consider student time, structures that are easily followed and designed for student success. And students really wanted to encourage feedback on the instructions, diversity of materials, and finally, accounting for the type of course delivery. So I think in a post-COVID19 world, they talked about the importance of in person and hybrid and online themes. So that was the last theme that we saw in that regard.

Rebecca: When you quote students, you include references to their identities. And when faculty first start teaching, they sometimes assume that students in their classes are maybe just as they were as a student, but the mention of the diverse student identities in the book, hopefully help [LAUGHTER] remind faculty about the diversity of their students, and may also help encourage faculty to use a version of the “Who’s in class form.” Can you talk a little bit about that?

Derek: So with Inclusive teaching, I think identity is a key word to be keeping in mind, this concept of identity, this concept that everybody that enters the classroom has their own unique identity that brings with it a wealth of assets, but also potentially some personal bias. And this goes for the instructor just as much as the students if not more, and with the role the instructor is playing, it might be most important there to acknowledge it. So why we include the identity information that our student respondents shared willingly within their surveys was to that reason, for some of our faculty, for our readers, for our instructors, to acknowledge that the students that are responding are not identical to us when we were students, or us as we are now. But also beyond that, to acknowledge that no two students are the same. And we’re seeing different identities across these student respondents. And with all of this in mind that even the idea of teaching to the average student, well, there is no average student really, and that creates significant issues, to the idea of inclusive teaching. So for all of these reasons, we did think it was really important to include those identity aspects of the quoted students that was shared. We think this helps frame the quote, with some context, some important context, and hopefully also makes the quote more real for our readers. This came from somebody, a real person, a real student. So I do think, to your point about the “who’s in class form,” and things like that, that once we as instructors recognize and are aware of the diversity in our student populations, it becomes much more clear that getting to know our students is critical to teaching inclusively, and to including them in the learning process. So you mentioned the who’s in class form, which was part of the first book, and that’s certainly one way to do it that we’ve had a lot of success with. I’ve personally used it for years now in every class that I’ve taught since its development, and had just great personal experience and positive feedback from the students. There was a study that was published that showed student perception and instructor perception also very positive for the who’s in class form. So it’s a tool that we share, because we want it to be available for instructors to use, they don’t need to reinvent the wheel in this situation. It’s a great way to start from the very beginning from the first day like Khadija was mentioning with that first assignment being important… also, that first day, that first interaction, that first relationship building step to be one where it says, “I want to get to know you because you matter in this classroom.” So that’s one tool that we offer up to our readers and want them to be aware of, but there are other ways to do it as well. And I think that it’s important for instructors to think critically about their classrooms and their institutions and their contexts and determine what works best for them within their class to get to know their students.

John: You mentioned the importance of the context for instructors. And one of your chapters in the book focuses on providing advice for historically excluded instructors and their allies, because faculty are often in a very different context in terms of how students perceive them and how students perceive their identities. Could you talk a little bit about some of the things that you emphasize in this chapter?

Tracie: Yes, perfect segue. So in this chapter, we really wanted to emphasize that we see you, we hear you, like understanding instructors from historically excluded backgrounds can have different experiences in the classroom, as you mentioned, John, a bit, but also to give some advice and encouragement and support for how to still do this work because we know that a lot of instructors even from historically excluded backgrounds, in even my experiences and me being one [LAUGHTER], we know that there’s a lot of investment in this. There’s a lot of investment and then the importance of this. So some of the advice we give is general advice. What we know that students really appreciate in general and then also some really concrete advice to really support them in having the evidence that they are teaching well. [LAUGHTER] So the first thing to really think about and we talk about in this section is just care, like, how do we demonstrate care for our students, and regardless of whatever identities, care is critical. We also talk a little bit about other types of strategies. Some of them may or may not resonate with folks, again, all of the stuff we give as options to really think about, to experiment with, to practice with, to really think about what makes sense for you. So we give a couple of examples from people who’ve actually been doing this work as well. So things that have been published around storytelling, thinking about your story and who you are, what you choose to share with your students. And you might withhold some things, you might share some other things, and there’s good reasons for that, depending on your identity as well. We also talk a little bit about engaging your allies and advocates, because you really want to have people in your departments, and at your institution, who really value and can speak highly of your teaching, they know that you’re doing a good job, and they can advocate for that. So engaging them in various ways is important. We have these great tools we talk about, the PAITE, etc. So being able to actually have data that actually shows that you are doing things that align with what the research and evidence says can support students and their success in the classroom. So collecting that information, and just making sure you have it when you need it. And just for your own practice is also really critical, I think, for instructors, in this way, too. So those are just a few of many types of guidance that we do give in this section. I think it’s really important to consider what makes the most sense. I will say, in working with instructors, and also thinking about myself as a black woman teaching as well, it’s so contextual. It’s just like you can’t say that this will happen in this course, or this or that with this instructor. But we know that there’s a lot of variation that can happen. And I think a lot of these different guidance points can really help equip people to really think about what makes sense for them in terms of how they engage with Inclusive teaching practice. And the last thing I’ll share is that even in What Inclusive Instructors Do, if we break down and disaggregate the instructors a bit, we see a lot of instructors successfully talking about doing this work from historically excluded backgrounds. And so we know like it’s happening, we do it in our classes, as well. So this is not being meant to be like discouraging at all, but a very encouraging chapter. And just having some tools and equipment to kind of continue so that you also can be seen that way at your institution and you have that information.

Rebecca: I love these themes of reflection and data and being able to tell your story of who you are as a teacher and why you’re doing these inclusive practices and that there’s evidence that it’s working and supporting your students. Chapter 10 of your book focuses on a topic that we’ve certainly talked about a bit on this podcast, and it’s all over the news and conversations in higher ed, which is AI and instructional practices. Can you talk a little bit about some of the ways in which generative AI can support an equitable learning environment?

Derek: I think it’s a really important topic right now. And on the ground, what I hear most commonly instructors talking about, and faculty talking about are the challenges that AI presents us around commonly topics like academic integrity, like, “Oh, here’s my assignment, my students are just going to go to [insert AI platform here] and complete that assignment and then submit it. And it’s hard for me… the tools for understanding if that occurred or not, can be challenging still at this point.” But I think what our chapter wanted to do more than focus on those challenges is kind of flip the table on that and focus on what are the potential benefits, like you mentioned, to increasing equity in the classroom and inclusivity in the classroom? And they are manifold, when you take the time to actually look at it and think about it. And I think the chapter goes over a number of examples to show how this can be done. In terms of benefits that we see AI being able to be leveraged with in the classroom, incorporating and developing more efficient learning experiences is certainly one of them. It can kind of balance prior knowledge gaps that students come with to the classroom by being a tool which can be used for brainstorming and ideation and things along that line, so that maybe a student can go to a platform with a particular question or broad topic and use that platform as a tool to help kind of narrow down something that they want to explore more deeply, get some ideas that maybe they didn’t have the knowledge that they brought with themselves to class to come up with that idea on their own, but now they can get there, and it can kind of even up that playing field a little bit. There’s also additional benefits to multilingual learners in terms of helping learn the language that their course is being taught in, writing more fluently and efficiently, and better understanding written material that’s present. It can help personalize learning in a really equitable way. There’s a portion of the chapter that speaks to the idea of using AI as an additional support for overcoming kind of the muddiest points in a classroom. So if an instructor is either directly asking students what is the most challenging topic, what is the muddiest point, those points could then be used and imported into an AI platform to help navigate those situations, or it could just be pointed as a tool that students could use on their own when they come up with their muddiest point and when they find a topic that’s challenging for them. Now, of course, all of this has to go within the concept of how reputable is the response that we’re getting, how trustworthy is the output of these different platforms. But that also actually leads into what a really potentially beneficial assignment tool would be with this, which is the idea of using these platforms, seeing what their responses are, and then having the students kind of fact check and revise and edit and think critically about those responses. And there’s an assignment there where students can essentially take a prompt, import it into an AI platform, see what they get back, and then the assignment is actually go through this, revise this, edit it, fact-check it: “What would you change?” And that can be a really great way to enhance the critical thinking and evaluation aspects that we’re really trying to build in a lot of our students with, again, narrowing and balancing what’s needed for kind of background knowledge coming in. So there’s these and many other mechanisms that are talked about in the chapter, including ones for, as instructors it can be a tool to brainstorm ideas for: “What are the most inclusive ways I can teach on this topic? Give me a list of 10 of these?” And then the instructor can look through that list and say, “Hey, which one of these might work best for my class context? And how can I adapt that to work for me?” So there’s lots of ways in which AI can be a really positive tool, as long as we use it in that way, and are appreciative and recognize the potential challenges that it also faces.

John: It’s certainly a bit of a challenging time in terms of professional development for faculty, because as you said, so many people just say, “this is just another tool for students to cheat,” because they haven’t really played with it that must themselves, they haven’t really seen the potential. And I think that’s something that all teaching centers are going to have to continue to work on quite a bit to help faculty become more aware of how these tools can be used to complement other forms of learning, rather than as a substitute, which is what they’re concerned about. So in addition to your two books, what are some other strategies that faculty can use to help create a more inclusive environment in their classes? I know, you’ve talked about some of them, like the “Who’s in class form,” and so forth? But what are some other ways that faculty can work to become more inclusive? Or what are some of the things faculty who have not thought about this much should start focusing on most immediately?

Khadijah: That is a great question. I think, yes, we have a ton of great resources, like books and tools like PAITE, and our lived experience. There’s something that I really think this particular study and process and talking to students that really resonated with us is thinking about nonverbal communication. Often people think inclusive teaching takes so much effort, but even thinking about your personality, you’re smiling. So for example, smiling, warm and welcoming tone, being thoughtful about tone. For example, students say not getting angry. So to throw back to something that Derek mentioned, is thinking about your instructor background, and what type of strategies or the nonverbal communication are you communicating. So that is definitely easy if you’re in person. [LAUGHTER] But if you’re online, some things that I think you can think about is the use of thumbs up, clapping, things like that. Something that recently came up for discussion was when you are online courses, do you tell your students to change the tone of the thumbs up so that everyone is reflected in that space and welcome? Students have actually responded that they feel more included in that space with nonverbal communication. So using emojis in emails, some students actually thought that made their instructor more approachable, but I will say, that being an instructor in 2024, you have to watch that. There is a pitfall there. Studies have shown overuse of the emojis, induce sometimes less confidence in the instructor. I think, being mindful of any strategy that you can use to be mindful of the nonverbal communication. But the verbal communication I’ll say, unequivocally, I keep going back to this… [LAUGHTER] the mug… it is really important to know names, say names correctly, and to use chosen names. So name tents can help with that, using an annotated class roster, seating charts, photos, mnemonic devices, student introductions, those things are concrete. And I think the last thing I would say, is actually telling jokes. So a lot of instructors use humor in college and university classrooms. So that tends to go over very well, there’s a positive effect on students in the learning environment, and making people feel included. But again, a word of caution, thinking about how the joke can land. So it could be very powerful in that way. But also wanting to be mindful if you’re going to, in fact, use that particular strategy to make your students feel welcome.

Rebecca: Well, we always wrap up by asking: “What’s next?”

Tracie: I’ll share a little bit about some of the things that I’m working on. So with regards to my scholarship, I kind of hover between teaching and learning and also educational development in general. And so a couple of things that are exciting, I think, for the future is we’re currently proposing a multi-site PAITE study. And so that study will dig deeper into the impacts of, we’re hoping, of the PAITE in the classroom. Since now, we’ve been using it a lot, but we don’t have all the data from the classroom and the experiences of students and the instructors. So we’re going to be doing a multi-site study in the next year or two. So hopefully, you’ll see that come out. Another project that we actually just finished off, a smaller project, we talked about group work, students experiences with group work, and that being a challenging area. So we implemented an intervention this past year at my previous institution, in collaboration with another center, and several classrooms, several instructors in their classrooms. And so we did collect some data on that. We just are synthesizing it. So we’re gonna hope to write that up and also share the resource that we use, that was an intervention that we created that students use to help empower them to really have more effective group work experiences, more inclusive, more equitable kinds of experiences. And then a couple of things I’m working on in an educational development realm, we just collected data from the POD community and outside on educational development right now in our times, so we can track how it changes. So we’re in the process right now, this summer, of starting to write the book, which will be the next one… it’s written like almost every 10 years educational development in the age of… and we’re gonna ffill that in [LAUGHTER] based on the results of the study, and then some other educational development things. And lastly, as was mentioned earlier, I’m moving to a new institution. So I’m going to launch a new Institute for Teaching, Learning, and Inclusive Pedagogy. So that’s going to be very exciting to really think about what’s next there, it will be launching the first center at Rutgers University, New Brunswick, and it will have embedded within it all of this inclusive stuff. So there’s so many things to bring there and to increase and expand.

Derek: Yeah, for me, I mean, I think coming out of the work of this study that led to this book, I think where I am is continuing to spread the word, but then also putting it into action. And that means personally, for me, in my own teaching, taking all the things that I learned through this process of listening to students, working with my fantastic colleagues, and writing this book, and having conversations like these, putting that into action, and making sure we’re walking the walk, so to say, because that’s the only way this actually happens. And in my role at the University of St. Joseph, again, I’m not only a instructor, a faculty member in the biology department, but I’m also the director of the first year seminar program here at USJSJ.. a two-semester long program. It’s a true first year program, where the first semester is a three-credit course that is kind of half transitioning to college and half an academic component where the students get to dive into college learning. And the second semester is a one-credit course where it’s service based and meeting the mission of our university to engage in compassionate service. So as in my director role, I am thinking a lot about how do we help students transition to college in their first year? And I think there’s a lot that we can gain from listening to students. And that was reported out in this book that can be leveraged there. So building in some of these ideas of really getting to know the students, engaging in opportunities for student choice within our first-year seminar program itself as well, kind of coming out of the work of this, I’ve created a college success portfolio that we’re implementing in the program. We implemented it for the first time this past year that ties a lot of these features in that allows students to talk about right off the bat, their first assignment is nothing to do with the content, but it’s the “Why are you here? Why college? What are your goals, both in college and out of college? And this is something that the students can succeed with, right away to kind of set that form and let them know that their instructors here at the institution care about them as learners, and that the learning that they’ll go through will be personalized for them. So something that we’re piloting here at the University of St. Joseph trying out but long term would be something that we would hope to share out with the greater community and hopefully help those first-year experiences at institutions around the nation and globe.

Khadijah: Well, for me, there’s one thing that really struck me as we worked on these two projects, was that inclusive teaching is very similar to inclusive research mentoring. And so I do a lot of research mentoring. There’s a lot of teaching strategies in dealing with different types of learners in a research laboratory setting, so I’m really interested in going in the direction of thinking about how some of these strategies and things that we learned translate to a research mentorship context. And also, because I’m at an institution now that has graduate students and postdocs, really train the next generation of inclusive instructors, and particularly those that are in STEM spaces. So when I’m working with a mentor to our postdoctoral advisory committee here, and trying to get them excited about being inclusive instructors as they transition to their next phase.

Rebecca: We’re looking forward to hearing about all the things that you’re continuing to do because you’re making an important difference in our teaching and learning community. Thanks for joining us.

Khadijah: Thank you.

John: It’s great to see all of you again and talk to you again. And this time, we shouldn’t let it go quite as long as between our last discussion and this one.

Derek: Thank you so much for having us again. It’s been a pleasure.

Tracie: Yes. Thank you.

Khadijah: This is so fun. [LAUGHTER]

Rebecca: Either that or you’re enough to write books faster, one or the other. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

351. Extending Kindness

Beginning faculty often receive warnings that lead to antagonistic relationships with their students. In this episode, Cate Denial joins us to discuss how a pedagogy of kindness can build productive learning environments for all students.

Cate is the Bright Distinguished Professor of American History and Director of the Bright Institute at Knox College. She is the winner of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award and sits on the board of Commonplace: A Journal of Early American Life. She is also the author of A Pedagogy of Kindness, one of the first publications in the new Oklahoma University Press series on teaching and learning, edited by Jim Lang and Michelle Miller.

Show Notes

Transcript

John: Beginning faculty often receive warnings that lead to antagonistic relationships with their students. In this episode, we examine how a pedagogy of kindness can build productive learning environments for all students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by

John: , an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Cate Denial. Cate is the Bright Distinguished Professor of American History and Director of the Bright Institute at Knox College. She is the winner of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award and sits on the board of Commonplace: A Journal of Early American Life. She is also the author of A Pedagogy of Kindness, one of the first publications in the new Oklahoma University Press series on teaching and learning, edited by Jim Lang and Michelle Miller. Welcome back, Cate.

Cate: Thank you.

John: Our teas today are:… Cate, are you drinking tea?

Cate: I am. I am drinking Harney and Sons Decaf Earl Grey.

John: …very nice.

Rebecca: It’s a great choice. I have a green tea today.

John: A what tea?

Rebecca: I don’t know it’s just green tea. [LAUGHTER] Sorry.

John: And I have a peppermint spearmint blend today.

Rebecca: Sounds nice and calming. So we’ve invited you here, Cate, to talk about the Pedagogy of Kindness. I think we last talked to you in October of 2021, when the book project was just getting started. And our discussion focused on a “Pedagogy of Kindness,” an essay on the topic that you posted in I think, August of 2019. This essay was such a great resource that helped so many people and I know, our community here was really interested in the subject. Can you talk a little bit about how this essay evolved into a book?

Cate: So, as I wrote the essay, there was so much I wanted to say, and I had to sort of just express it in the most concise form that I could. After I was done, I was sort of pondering the other things that I wish I’d had the space to be able to include. And then Josh Eyler, who is now at the University of Mississippi, messaged me and said, “Hey, I think this needs to be a book.” And I was like “Me too.” [LAUGHTER] So I started thinking of what I would do to make it a longer treatment. And I wrote up a proposal, sent that off, and started writing, pretty much straight away.

John: In this book, you argue for a teaching approach based on kindness. You note that many people, though, confuse “kindness” with “being nice.” How do you define “kindness?” And how is it different from “being nice.”

Cate: So, being nice is a lot of what we do in academia, nice lies, nice will do anything to just get along, it will avoid conflict, and it will just try and paper over cracks in our institutions, whereas kindness is fundamentally honest. So it means you’re going to have some really tough conversations with your colleagues and with your students. You’re not going to mislead them about how they’re doing in your class. And kindness is really about accountability, it’s about positionality, and it’s about responsibility. And those are all things that I think niceness just elides.

Rebecca: I know that I found myself saying many times after reading your essay: “Telling the truth here or letting a student know what their real situation is, would be the kind of thing to do. [LAUGHTER]

Cate: Exactly,exactly. And I often have people, as I go around the country and talk to other faculty, ask me: “Exactly how do you have these conversations, when the student comes to you and says, ‘this terrible thing has happened, and please just excuse this paper,’ or ‘don’t include this in my grade’…and you have to, especially if you are in a professional school, for example, or a medical school or something like that.” And that’s when that distinction between niceness and kindness really comes into play. Because niceness is just about getting along. And kindness is like, “Well, let’s have a real conversation about what’s going on here. Why these expectations are, what they are, whether it’s possible for us to make some kind of adjustment here, or whether this is a moment where we really have to talk about what your future is in this course, or in this program,” things like that.

John: In the introduction to the book, you note that this is not quite how you started teaching, that you started teaching, like many of us, with a somewhat different focus in terms of interactions with students. Could you tell us a little bit about your perspective when you first entered teaching?

Cate: So when I first entered teaching, I was brand new to teaching. I had got my degree in England one month before I stepped into a classroom in the United States. I did not know the United States educational system well at all. I felt like a complete fraud, and the limited training I had had at my graduate program really encouraged me to think of my students as antagonists, to assume they were going to cheat, they were going to try and get one over on me, that they wouldn’t do the reading, that they were going to try and negotiate for a grade that was not what they deserved to get, things like that. And walking into a classroom like that prepares you for battle [LAUGHTER] at every single moment, which is a terrible way to walk into a new relationship of any kind. It’s especially demoralizing and damaging, I think, when you are walking into a classroom, so I had to unlearn the business of thinking that my students were up to no good. I had to unlearn the business of thinking that my students were fundamentally not interested in their own education.

Rebecca: You mentioned your four day visit to a digital pedagogy lab, and how that impacted your teaching. Can you elaborate a little bit on that and share that story?

Cate: Yes. So I went to the Digital Pedagogy Lab Institute in 2017, which was in Fredericksburg, Virginia. It was the most incredible experience. I went because I had the feeling that I was lagging behind in bringing digital tools into my classroom. And I thought this would be where I would learn those things. And I certainly had that opportunity. But the first thing that I was asked to do in the intro track, was to tear apart everything about my teaching, in a really good way. I was in a track that was led by Chris Friend and Sean Michael Morris. And they asked us to reflect upon everything about our pedagogy. Why were we doing things in the way that we were doing them? What was the rationale that we had for the choices we were making, and the way we addressed our students and the kinds of things that we have happen in our classrooms. Everything about that institute was predicated on kindness. And it’s not that someone actually said at some point, “Why not be kind?” But that was the subtext from everything from… it was the first Institute I think I ever went to where there were pronoun pins that people could take when they registered. There was really generous swag, there was this wonderful ethos of sharing, everybody rolled up their sleeves. and really worked hard at turning everybody’s teaching materials into something even better than what they arrived with. There was really no hierarchy. It was just a wonderful community experience of trying to get better at what we do.

Rebecca: Sounds like such a great experience, and an opportunity to really critically reflect on your teaching, which we don’t always take the time to do.

Cate: I didn’t even know I needed to do it. [LAUGHTER] So by the time I went in 2017, I was more than 20 years into my teaching career. And I thought I was pretty great at teaching at that point. But what the digital pedagogy lab made me realize was that no matter what my goal was, as a teacher, all of the documents in my course, my syllabi, my assignment sheets, the way I handled introductions, all kinds of little things, really told a different story. And they told that story of antagonism that I had been socialized into as an early instructor. And I think that is something we are socialized into globally in higher ed.

John: One of the things you talk about is finding out a little bit more about your students. Because when many of us start teaching, we tend to assume that our students are similar to us, they have similar backgrounds, they’ve been prepared in ways that were similar to ours, maybe they weren’t quite as good as students as we were. But many people start off by assuming that students will be just like them. And yet you note in your introductory chapter that as many as 71% of undergraduates in the United States are non-traditional, under a teaching approach that focuses on kindness, how might faculty begin to address the diverse identities and needs of our students?

Cate: So, I think the first thing that we need to do is take stock of who our students are. And so we need to think about the demographics on our campus, that information is usually pretty easily available. As I travel around the country and visit different campuses, I always pause and look up on their websites, what is their diversity rate? And how many low-income students are there? How many people get Pell Grants? How many people are first-gen? How many people are returning after taking some time off? Or maybe have a GED? There’s all kinds of ways in which our students are not that sort of stereotypical 18 to 22 year old, straight white student who is perhaps in a sorority or fraternity or does athletics or anything that sort of gets caught up in the stereotype of who students might be. I’m really interested in what happens behind that. Even if a student walking into my classroom is, for example, a straight white student who is 18 to 22 years old, or maybe is an athlete, maybe is in a sorority. That doesn’t mean that those things tell me who they really are. They don’t tell me what is going on in their home life. Do they have kinship obligations and responsibilities? What is happening with the way that they’re funding their education, all kinds of different things. So I think pausing at the beginning of the term and really getting to know who our students are, is vital if we’re going to meet them where they’re at in our classrooms. So there’s lots of ways of doing that. There are ways of sending out surveys to our students, either before the semester begins or right as it begins. There are ways to have conversations in class to make those introductions a little bit more substantial than just, “Hi, my name is….. And my major is….. , and I’m taking this class, because….. So trying to gather that information, and also reflecting on how my identities are showing up as I walk into a classroom. I think those are really important parts of what we do.

Rebecca: What are some of the ways that adopting a pedagogy of kindness changes the ways that instructors and students interact?

Cate: I think it is all about undermining that presumption of antagonism. If we do not assume that our students are out to get us, But rather, that our students, in the vast majority of cases, are actually really invested in being there, and are perhaps a little bit scared of us, that they are a little bit uncertain, a little bit unsure, maybe they are first years, or this is their first class in a certain discipline, or they’ve been out of school for a while. There’s all kinds of ways that they might be very, very nervous coming into our classrooms. And so I think, taking the time to think about that, and address that, and make space for that, so that our students are not some sort of terrible, antagonistic, cruel set of people who really don’t want us to succeed or for themselves to succeed. That’s just not based in reality.

John: And I would think it would be a much more enjoyable environment, both for students and for the instructor, when there’s that trust between students and faculty.

Cate: Yeah, I actually have a blog post that I wrote, I think back in 2016, called the first-day jitters, that was about how nervous I would get before I would meet my students for the first time every semester. And once I changed my outlook and stopped expecting antagonism, I stopped getting nervous. I walked into the classroom and was excited about what was going to be co-created in that space. And it made an enormous difference to me and to my students. Certainly, I saw it in the ways that they interacted with one another and with me in our classroom spaces, but it changed everything for me about showing up as a teacher.

Rebecca: One of the things that you’re pointing to or hinting at is the idea that the pedagogy of kindness also applies to being kind to yourself. Can you talk a little bit about ways that instructors can be more kind to themselves?

Cate: Yeah, I think the number one way in which we can all be really kind to ourselves is to have impeccable boundaries. [LAUGHTER] I think it’s really important that we not prostrate ourselves, not give beyond what we are capable of giving, and have a really good understanding of what our capabilities are. So for example, if I am someone who is stringing together three jobs to make a livable income, if I have a family, if I have caregiving responsibilities, if I have a really long commute, all of those things are in the mix. And I cannot then be someone who 24/7 is simply available to my students. I don’t think that’s healthy, even if you don’t have any of those other responsibilities or considerations. I think that we all need time to have downtime, to have rest, to have play to, have pleasure. But I think that we need to be able to say, “I am available by email between these hours, you can expect an answer from me within a certain period of time.” Communicate those things incredibly clearly to our students. And that means that we’re also modeling for them that it is okay to take time for yourself and to have good boundaries.

John: In the second chapter of your book, you focus on the syllabus, which is often one of the first forms of communication between the instructor and students. Quite often the syllabi look more like legal documents than they do a welcoming introduction to the course. What are some of your suggestions on how to demonstrate kindness in your syllabus?

Cate: So an exercise that I was asked to do at the Digital Pedagogy Lab was to read my syllabus with a critical eye and ask myself who I was addressing when I wrote it. And what I found was that I was addressing someone who I assumed was going to screw up somehow. I assumed that they were going to disappoint me they were going to let me down, that they were going to try and get away with something… that old antagonism rearing its head. So then I had to stop and go, “That’s not who I think my students are, so why is that reflected in the language I’m using in my syllabus?” So then I stopped and I rewrote. And I kept in mind a student who is very real, I pick students to have actual students in my mind when I’m writing my syllabi, or when I’m revising them. And in addressing those students, I was able to make my language welcoming and warm. It doesn’t mean that it’s soft, it doesn’t mean that I don’t have boundaries and that I don’t have expectations. It simply means that I’m talking to them as I would if we were in person. That’s one thing that I learned to do. Another is that I learned that for many students, syllabi are impenetrable walls of text. And I’m sure all of us, in our time, if we haven’t written that syllabus, have seen that syllabus, maybe when we too were students. Those are really hard for students who have things like ADHD, for example, to be able to parse, to find the important information. They don’t look welcoming. They look like legal documents. And so I’ve changed my syllabi, for example, to have really colorful welcoming headers. I make a point of always saying “welcome to the class,” as the first thing that you see on the syllabus. I make a point of emphasizing relationship early and saying, “This is how you can get hold of me. Here are what my student hours are and what student hours mean.” And only then do I get into the stuff like, here’s what we’re going to read and here’s the assignments we’re going to do. It’s all about establishing warmth and a relationship from the get go.

Rebecca: How do you balance that approach with some institutional policy or other things that you maybe have to include in a syllabus that don’t have that same tone and tenor?

Cate: Yeah, that’s a really big consideration for a lot of people. There’s a lot of boilerplate language that has to go into many syllabi. So there, I borrow from Remi Kalir, who has students annotate the syllabus, and I’ve been doing this for several years. Now, my students get the syllabus at the end of the first class session, and they take it away for homework and they annotate it. I give them a really simple definition of what annotation is. And then in our second class period, we come back and they get to tell us what their questions were, what their comments were, what seemed unusual to them, what really stuck out and we have a conversation about all of that. And if you have a bigger class than I do, this is the sort of thing that you can have small groups do or you can have parents do. And then you can have people report out some of the things that they thought were most pressing. What that does is it returns to students some agency, and they can comment on the language shift, the tonal shift between the things you wrote and the things that you have to include, and you can have open conversations about “So why do you think we have to include this stuff? And what are ways that we could say this differently?” or “What is the way that you understand this best,?” Having that conversation is really important in allowing students to talk back to that document. And to make it clear that it’s a human creation, that it’s not just spit out by a machine somewhere, that there are people behind it, and that we are collectively always polishing and revising our syllabi in light of these relationships that we have on our campuses.

Rebecca: In smaller classes, I’ve done the annotation in a single document. And it’s been really interesting to have students comment on each other’s comments in a digital format, too. And it really brings some interesting things to light.

Cate: That’s a great way of doing it. I tend to have my students do it individually, just because they don’t know each other very well yet. And I found that my students tend to respond better to individual documents, and then a group discussion, but it works differently with every single instructor and every single set of students. So there are multiple ways that people could do this kind of annotation.

John: You did mention that this syllabus is not something spit out by a computer, but last week, we had a workshop where one of the things I suggested was that faculty may want to submit their syllabus to one of the AI tools out there and have it checked for inclusive language or to rewrite it in a more inclusive manner. And I think a lot of people found that, in that case, the computer did a better job than they did [LAUGHTER] in terms of creating that inclusive syllabus. But, at the other extreme, one of the things I’ve often done and I think you mentioned this in your book is in smaller classes, co-create the syllabus with students at the very start of the semester.

Cate: Yeah, I teach a course on history pedagogy. And there I give my students blank sheets of paper and say “Here’s your syllabus,” [LAUGHTER] and then they get to start brainstorming the things that they should put in syllabuses. They’re experts at syllabi, they read them all the time, they see many more than most of us do. And so they brainstorm what they think it should include. And then I put them in small groups. And together, they start to talk about, “Oh, that was a great idea you came up with, I didn’t think of that. Let’s put that in our syllabus.” Then we write their syllabi up on huge post-it notes that we put on the wall. And then everybody gets little post-it notes. And they get to walk around, and I ask them to do what I was asked to do at the Digital Pedagogy Lab, which is to say, “Who is the student that these authors are imagining as they write the syllabus?” Write it on your little post-it note, put your reactions on the wall. And what they very quickly learn is that they too have absorbed the idea that students are antagonists, students can’t be trusted, students are going to mess up somehow. And so then, as a class, we sit down together, and we say, “Okay, we don’t want to do that. We don’t want to make those assumptions. So how are we going to write a document that accurately sets out expectations, that also supports and that communicates trust?”

Rebecca: In chapter three, you focus a bit on assessment. And many faculty note that when students do less well than maybe they thought they were going to or anticipated on a graded activity, they might become disengaged or lose interest or disappear. Can you talk a little bit about ways that we can make our assessments more equitable, and to make expectations more transparent to our students?

Cate: There are so many ways that we can do that. Including students in the process at all points, I think, is a way that we can never really go wrong. But I think that one of the most important ways we can approach assessment is to think about, are we being inclusive? are we giving our students the tools they need to be able to show us what they know?…because so often, what we’re actually grading or assessing or giving feedback on is, how they write about what they know, or how they make an equation about what they know, we’re not actually just finding out what they know, what they have learned. So giving them multiple ways to express what they know, is a great way of leveling the playing field of allowing students who maybe don’t excel at writing to still be able to show “Hey, I have been thinking, I’ve been turning this over and over, I have some new appreciation for whatever it is art, or music, or fishbones, or an equation, or set of chemical compounds.” So in my classes, it’s traditional for history to be assessed through writing papers. And I certainly do have my students do written work, I don’t get rid of it altogether. But I also make an assignment where my students get to create projects. And they can do anything they want to, except write a paper. So they do things like make quilts, and food, and board games, and they knit and make pottery. I’ve had people make raps and sea shanties. I’ve had people dress up as pirates and come talk to us about a day in their life. All of these things are based on historical research. All of these things have a reflective component at the end. But for a student who really struggles with writing, for example, this gives them a way of being able to show me they have been learning that term, that they have done research, that they’ve really engaged with the class material in a way that doesn’t limit them in any way.

John: In chapter 4, you address kindness in the classroom. Could you share a couple of ways in which faculty can display kindness in their classroom interactions?

Cate: I think one of the most basic and important ways that we show kindness in the classroom is we learn who our students are. And that means names. And that is increasingly challenging. It depends on how many, if you’ve got 300 students in your classroom, how do you learn their names? I tend to have smaller classes. So it is really doable for me to learn their names, but I still rely on tools that helped me do that. So I bring in name tags, and for the first several sessions, my students will write their name on their name tag, and if they feel comfortable, they’ll put their pronouns on there too. I know that Vigi Sathy and Kelly Hogan talk about having name tents in their big lecture classes. And those name tents are also color coded. So then that helps them put people into small groups. People can turn those name tents on their end when they want to indicate they want to participate in conversation. They’re wonderful, wonderful tools. So there’s lots of resources out there that sort of suggests there are many, many ways to make sure that we’re getting to know our students, but I think that’s a really basic thing we do that says, “I know who you are. I know when you’re here, I really appreciate you being here and being engaged in a classroom activity.” I also think that it’s important to teach our students how to be in that space together. I think this is really important coming out of the height of the pandemic. We’re not done yet. but we are coming out of those most acute years. And certainly I have seen among my students on campus, that there was a period of time where it was odd for them to interact with other people because they hadn’t had practice. They had been in lockdown. They had been at home. They had been doing school online. And so we really had to sort of think about how do we interact? How do we talk to one another? How do we show that we respect one another? When something goes wrong, how do we show that we are accountable and responsible? How do we apologize, and mean, it. These are all things that we need to teach, as instructors to make sure that our classrooms are kind spaces in which learning can really happen.

John: Is there anything else you’d like to share with our listeners that we haven’t yet discussed?

Cate: One of the things that comes up in the conclusion to my book is that it’s very easy to feel tossed around in higher education, like we’re in tiny little boats on a very large and angry ocean.[LAUGHTER] And certainly, there are structural problems that need our attention in higher ed. But we’re not powerless. And we get to make changes in the spheres that we control. And we get to approach our students, as fellow human beings who matter, we get to treat ourselves as people who matter. And kindness is the fabric of that, it’s what weaves everything together. It is the material from which we build inclusive spaces, and spaces where people’s words and ideas are valued. And without kindness, we end up with places that are disrespectful, we end up with places where people do not feel like someone’s listening to them. So I have the metaphor, that little boat being tossed around in my conclusion, but ultimately say, if we can just be kind to one another, then we can sort of temper that storm.

Rebecca: It’s a great image to end on, for sure in your book. And it’s such a great metaphor for really thinking through what sometimes it really feels like to be in higher ed, not only with our students, but with other colleagues. There’s lots of ways to bring this kindness work into other spaces within higher ed outside of the classroom as well.

Cate: Yeah, I agree. And one of the projects I’ve been involved with over the last couple of years is something called Care in the Academy, which is about showing kindness and care to each other. So focused on faculty, focused on staff, focused on those relationships, and how we show one another that we matter to each other.

Rebecca: Can you share some examples about what that might look like in spaces outside of the classroom?

Cate: Yeah, so some really concrete things that we recommended after 18 months of really thinking critically about this issue were things like extended bereavement leave. Places tend to give us three, maybe five days and they’re immediate. And the idea is that whatever mourning or immediate grieving you need to do will happen in like a week. And that’s just not practical or realistic, or how humans work. And in many cultures, it’s also not reflective of the way that people process death. So we talked about having maybe another three to five days that you can use for a year after someone passes and you can mark an anniversary, or maybe there is a ceremony that you need to attend. Or maybe there is a ritual that you need to perform, and you take time away to be able to do it. It’s a very small thing, but it makes such a difference in sort of saying we respect what a difficult time this is for you. Trying to improve reimbursement culture, there are so many people who are stringing things along on their credit cards, basically making loans to their institutions, and trying to speed up the process by which they are repaid as a basic form of respect, I think is really, really important to the way that we all function around one another. To not assume that people are sitting on vast mountains of cash that they can just make available to do things like go to conferences or to travel to another town or whatever the thing is. I think that’s also really important.

John: That’s especially true for junior faculty, for people who are contingent faculty and who are struggling a bit because this occupation does not pay all that well. It gets a lot easier once your kids are out of the house and you’re past those stages, but it’s a struggle for a very large share of faculty.

Cate: Especially given student loans being what they are. This lasts a very long time.

Rebecca: It’s something that sometimes faculty have in common with students and we don’t always recognize or acknowledge that.

Cate: Yeah, I agree. And I think our students don’t often look at us and think, “Oh, wow, they’re still paying off their student loans.” [LAUGHTER] They tend to see us as something other than precarious. Whether we are in a contingent position or not, very often our financial lives are in some kind of precarious situation, just because the reality of funding higher education careers is what it is right now.

Rebecca: So we always wrap up by asking: “What’s next?”

Cate: What’s next is that I am going to go read a book about biomimicry. And biomimicry is looking to the natural world for patterns that all organisms share, and taking wisdom from those patterns. So it helps us in terms of ecology. But I just heard a podcast that was applying this to the way that we organize our institutions, and thinking about the way that organisms heal after fire or clear cutting or a flood, for example, the different jobs that different organisms have in the recovery process, and thinking about how we make really resilient institutions and organizations by looking to the natural world. So I know very little about it. I’ve listened to that one podcast. So I am going to go read a book that is all about that topic.

Rebecca: That sounds really fun.

Cate: Yeah, I think so.

John: And I think we’ve all needed a bit of resilience over the last few years with the challenges we’ve all faced. So it sounds like an interesting way of addressing that issue.

Cate: Yeah, I’m hoping I will get some fresh ideas.

Rebecca: Reading stuff that’s unfamiliar is always a great way to reinvigorate our own work.

Cate: Yeah, and I have not really considered biology since I was 16 years old and took my GCSEs. So this will be a real treat and a delight to exercise part of my brain that hasn’t been exercised in a while.

John: Well, thank you for joining us. It’s great talking to you. And I really enjoyed reading through your book. It grew from this very nice, useful essay that many of us were reading through during the pandemic, to a much more elaborate discussion of how to be an effective teacher in all aspects of teaching.

Rebecca: Such a needed robust resource.

Cate: Thank you so much. It was such a pleasure to be with you today and have this conversation. And I really appreciate all your kind words about the book.

Rebecca: We look forward to talking to you in the future, hopefully the near future.

Cate: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

349. Growth Mindset Messaging

First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, Elizabeth Canning, Makita White, and William B. Davis join us to discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU.

Show Notes

Transcript

John: First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, we discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Elizabeth Canning, Makita White, and William B. Davis. Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU. Welcome back, Elizabeth and Makita, and welcome, William.

Bill: Thank you. It’s a pleasure to be here.

John: Today’s teas are:… Elizabeth, are you drinking tea?

Elizabeth: I am drinking tea. I’ve learned from past podcasts with you that I should bring some tea along. So I’m drinking some sweet and spicy black tea today.

Rebecca: Sweet and Spicy… [LAUGHTER] sounds interesting.

Elizabeth: It has some cinnamon and some orange. It’s very good.

John: Very nice. And Makita?

Makita: I’ve got classic Earl Grey today.

Rebecca: Can’t go wrong with a classic. [LAUGHTER]

John: And Bill?

Bill:I have a Raspberry Zinger herbal tea today.

Rebecca: I’ve moved on to Awake tea… this time of day. [LAUGHTER]

John: We recorded another podcast much, much earlier today. So that’s probably needed a bit more. [LAUGHTER] And so that I won’t be awake all night tonight, as I have been for much of the last couple of nights, I have just a pure peppermint tea.

Rebecca: Probably a better choice for this time of day. [LAUGHTER]

John: And for this time of the year.

Rebecca: So we’ve invited you here today to discuss your March 2024 study: “Growth Mindset Messages from Instructors Improve Academic Performance Among First-Generation College Students,” which is quite a mouthful. [LAUGHTER] But before we discuss the study, can you talk a little bit about the difference in outcomes between first-generation students and continuing-generation college students?

Makita: Sure. So when we say first generation, we usually mean someone who’s the first in their family to attend college. For this study, we specifically defined it as a student for whom neither parent has earned a four-year college degree. And these students typically have to face and overcome a lot of social and cultural barriers in order to be successful in college. And although almost a third of the people who go to college are first-generation, they don’t always seem to do as well as continuing-generation college students. On average, they tend to have a little more difficulty adapting to college, they typically earn lower grades and they have higher dropout rates. They also act a little bit differently, they’re less likely to do things like go to office hours, or communicate with their instructors, whether that’s by email or in person to seek clarification on course materials. And just in general, they tend to be less likely to try to access helpful academic resources when we compare them with continuing-generation college students, which would be students for whom at least one parent has earned a four-year college degree. And on top of all of that, the students are also usually… I mean, they have to be… dealing with having less familial guidance than their peers do when it comes to navigating higher education. So if you’re a continuing-generation student, you have a parent who might tell you something like, “Hey, it’s a good idea to go to office hours,” or “You should be asking questions” or “there’s a financial aid department,” important stuff like that. They also may be dealing with a mismatch between the values and culture that they grew up in, compared with the values and expectations of an American university, which is typically very individualistic and may not be as supportive as what they are used to. And of course, they’re also more likely to be working, living off of campus, and they usually get less financial support from their families. But there are other factors that may exacerbate the difficulties that they’re dealing with. And this study is about one of those.

Bill:And one of the things that, just for context at our local institution to keep in mind, is WSU has a lot of, for lack of a better term, brand loyalty. It was not uncommon for me in my class to have fourth generation Cougars. And if you think about that legacy of your great-grandparents had attended this institution, as had other people, that sort of familial knowledge that gets passed along, is a huge benefit for that population of students as compared to the first-generation students as Makita was talking about.

John: One of the things that you mentioned, Makita, was differences in help-seeking behavior. And one of the studies on that is one that we talked about with you and Elizabeth in an earlier podcast, and we’ll include a link to that in the show notes. Could you describe this current study?

Elizabeth: Yeah, so this study we conducted in 2021, I believe. It was the second semester online, so the pandemic year. I know we’re all sort of crossing out that memory from our minds… [LAUGHTER] …that whole year. As instructors, it was a little tumultuous. But Bill and I got together and we designed this study together in his class. He was teaching at the time the introductory biology class at WSU. And what we decided to do is test out some actionable techniques that instructors can do to communicate a growth mindset to their students. So a lot of the research that my lab does looks at how growth mindset messages impact different students from different backgrounds. And at this time, there was a lot of suggestions about how to implement growth mindset ideas in your class, how to change course material, how to integrate it throughout, but there wasn’t really any experimental evidence of specific strategies that instructors can do. So we decided that we would just test it out in Bill’s class. So what we did in this study is we randomly assigned everybody in the course to one of two conditions: a control condition or a growth-mindset message condition. And after the first two major exams in the course, students received an email from Bill, and in the control condition they received a message that is pretty typical of an instructor after an exam has been conducted in their class. It says things like: “exam grades are posted online, here’s how we calculated them,”
“here’s what it means,” and then “feel free to talk to me in office hours, or ask questions about it.“ In the growth mindset message condition, we included all of that information, but we also included a fairly lengthy paragraph… I’ve gotten comments on… but it’s a big chunk of a paragraph that basically says, in Bill’s words, “Here’s what I believe about student performance and exam performance and improving in the class.” And there we integrated growth mindset theory, we talked about how abilities can be improved, we talked about how academic struggles are normal to experience, and that this struggle is part of the process of learning, that is something that is controllable. That you can put in effort that you can use different strategies to make improvements. And I think we used, Bill, if correct me if I’m wrong, we started with a draft of this email that you had already been doing in your class. And we just sort of infused it a little bit more. So Bill really wrote like the first draft of it, and then we kind of went back and forth a little bit to edit it and infuse it a little bit more with growth mindset. But it’s really written in the voice of the instructor. After we implemented this email after the first exam, we did it again after the second exam, changing the language a little bit. We assessed their performance across the semester. So we looked at their exam grades, we looked at their final course grade, and we found that students who were in the growth mindset condition earned higher grades in the course. And this was predominantly driven by first-generation college students. So first-generation college students performed much better in the course, when they received those growth-mindset messages from the instructor.

Rebecca: Can you talk a little bit about the effects and how significant they were?

Elizabeth: Yeah, so when we look at the group differences in the control condition, what we find that’s very consistent across big STEM courses across the university, that continuing-generation students are getting better grades than first-generation students. So in this course, they’re earning about three-fourths of a letter grade higher than first-generation students. So this is a really, really big gap between the continuing-generation students and the first-generation students. It’s not specific to this one course. So we see this in a lot of different datasets. But what was really remarkable in this study is that that difference in their grades was completely eliminated in the growth-mindset condition. So first-generation college students earned about three-fourths of a letter grade higher in the growth-mindset condition compared to the control condition. So this is a pretty large effect on final course grades. And of course, it’s pretty significant in terms of the curriculum that they’re a part of. Bill, if you want to talk a little bit about the students who took this course and how it plays a role in their career trajectories.

Bill:Yeah, thank you for that. I was looking forward to jumping in a little bit. And this is a little bit about my journey too, in this course. So, at this point, just for context, this was the 16th semester that I had taught this course. Largely I had been either the sole instructor or I was the primary instructor of record for the course. It’s taken by about 450 to 550 students each semester, little bit of differential between fall and spring in size, over 50 different majors, or pre-major students take this class. It’s nominally a freshman majors biology course. But there’s a substantial number of students who are neither biology majors or life science majors. And there’s a sizable number of students who are not freshmen. So, in reality, the highest population of my course normally were sophomores, freshmen were the largest population in that spring semester. And that was pretty typical, because it is oftentimes taken as the second semester of an introductory biology sequence. So yeah, and to be honest, over the years before this study, I had been trying to narrow those opportunity gaps between first-generation and continuing-generation students. I knew they existed. I had done many things with the course experimentally over the years, and never had really closed that gap. This was one of the first times, if ever, I saw that gap completely close in anything that I tried, and I tried tinkering in the lecture in the lab, big changes, small changes, a lot had gone on before this.

John: You mentioned that the class is normally about 450 students. Could you tell us a little bit more about the specific course in which you implemented this? You noted that this was during the pandemic. Was this offered remotely? Was it synchronous? Was it asynchronous? And how many students were there in this particular sample?

Bill:So, the course was taught in a synchronous Zoom manner. So both the lecture and the lab were taught using zoom. I believe, if memory serves, I had the largest Zoom Room on campus that semester. I can’t remember the exact number, I want to say my enrollment that semester was somewhere close to 550, or 600

Makita: The final sample is 417, I believe.
Bill:417. That’s right. That’s how many students, so it must have been around probably 450, at the beginning of the semester, or something like that. Yeah, we had synchronous Zoom every day with students. In fact, I learned pretty quickly how to crash Zoom my first semester I taught remotely. [LAUGHTER] So by then, it was a little bit more of a pro, and at least knew how to keep it running for 50 minutes at a time.

Rebecca: And true badge of honor.

Bill:[LAUGHTER] Well, there are many lumps that we took in the pandemic and learning how to teach and Zoom was one of them. [LAUGHTER]

John: I had 350 students in the fall of 2020, and it was not my most enjoyable teaching experience. What are the factors you controlled for? Because you did control for some other variables to separate out the effect of the growth-mindset messaging?

Elizabeth: Yeah, so we controlled for three things in our analytic models. We controlled for students’ prior college GPA… so this was just their self report… GPA that they’ve had in college. Since this was in the spring semester, all the freshmen had had a semester under their belts at that point, to have an idea of what their college GPA was. We included this just because we were looking at performance. So we wanted to filter out or control for their prior performance, that’s a big predictor of their performance in the class. We also controlled for their self-reported personal mindset beliefs. So we asked the students themselves, what do you believe about the nature of intelligence? Do you think it’s something that can improve? Do you think it’s something that you’re just born with? So we measure their own self-reported mindset beliefs, and we control for that and all of the analyses, because we want to look at the effect of the intervention above and beyond student’s own beliefs about ability. And then the last thing we controlled for was their race or ethnic status. So what they self reported to us on a survey. Here, we controlled for that because there is some overlap between race, ethnicity, and first-generation status. We did look at the different groups separately, we don’t have enough power to really look at race ethnicity as a moderator at the intervention. So we control for it in our analyses to look at the effect of first-gen status above and beyond race and ethnicity,

Rebecca: I can imagine, as an instructor who’s fiddled quite a bit in their class to overcome these equity gaps, that it might have been a little surprising [LAUGHTER] how much of a change happened with such a relatively small intervention.

Bill:It was shocking. It’s funny, because one of the things that Elizabeth has really talked about is how we took materials that I actually already possessed in some ways, and modified them slightly in some cases, and more radically in others, but just that transformation was enough to start to really make an impact. And you could see it happening. And that was amazing to me. I remember when I halved my opportunity gaps in my class with an intervention. And I remember dancing and celebrating, [LAUGHTER] and then this one was just on that scale. This was huge and amazing to me.

John: One of the interesting things in this study is you were looking at the effect in terms of grades on subsequent exams, but you didn’t find much of an effect on the second exam from the first intervention. But when you had two of those messages coming out, that’s where you found the relatively large effect. Why do you think the second message was having more of an effect than just a single message?

Elizabeth: Yeah, so we have a couple of theories. We think that one reason could be that they just need the message more than one time. So they need two doses or more than two doses to sort of really get people’s attention and make them change their behavior. That c ould be one reason. Another reason is that it might be enough to just have one message, but it would take time for behavior to change in order to see a difference in their performance. So it could be that after the first exam, they started engaging with the material differently and then that then sort of snowballed to affect their third exam scores, and then again, their final course performance, but we don’t really know exactly what’s going on. Another reason could be that towards the end of the semester, students get a little antsy, and they start waking up a little bit to, “Oh, I should probably be doing something different if I want to have a grade that looks like this.” So it could be timing, it could be dosage, it could be the effects take longer to materialize in terms of performance changes. One of the interesting things that we found here in this dataset is we did try to look at behavior in the way that we had access to, so we pulled all of their activity on the course website. And this semester, our campus was using Blackboard as the course sites. And we pulled all of the Blackboard data. So we looked at whether students in the growth mindset condition were engaging with Blackboard differently. And we found that they were. Students, on average, who were in the growth-mindset condition engaged with the course material 40 instances more, on average, than students in the control group. This is around a 12% increase in webpage engagement. So after they get this message from the instructor saying that intelligence is something that can grow, abilities can improve over time, and here are some strategies that you might implement to realize those changes, they’re going to the course website, they’re clicking on things, they’re re-reading their notes, they’re looking over the course lectures, they’re engaging more with the resources that are posted there. And then not surprising to any instructor ever, [LAUGHTER] that when students are engaging with the course materials that then lead to greater performance in the class. So we found a mediation effect through their behavior with the course material that was posted online. So this is one example of the behavior change that I was talking about earlier, that when they get this message, it can be inspiring, it can inspire a different type of behavior that can lead to better performance in the course.

Rebecca: Those are really strong click rates generally, [LAUGHTER] especially when we’re often complaining that students don’t read our emails or messages. [LAUGHTER]

Elizabeth: Yes. So, we did look at differences in Blackboard in terms of clicking on course material versus going to the page where all their grades were, like the gradebook. And we didn’t find any differences on their engagement with the gradebook. So it wasn’t that they were just tracking it more or anything like that. But it seems to be the case that they’re actually engaging with the substantive course materials more, on average, in the growth- mindset condition.

Rebecca: It’s just incredible. [LAUGHTER] Can you talk a little bit about… and we’re hinting towards this, obviously… what does this study suggest that faculty and institutions can do to improve outcomes of first-gen students, or maybe even, Bill, what you’ve continued or started to do?

Bill:I think, for me, what I took away from it was being very intentional with my students in communicating what I believe about them. And I mean that in the spirit of the old NPR show, or the segment that they used to do, you know, “This I believe,” where people would have those small snippets. I felt like this intervention was something along those lines where I could be authentic in communicating what I truly believed about their abilities. And I did it in a way that, with some small tweaks, obviously had these much deeper and larger payoffs. And I think that’s one of the real nuggets of wisdom here is that when you want to transform, and have better student outcomes, you want to narrow these opportunity gaps for students from certain backgrounds or experiences when they come in. Oftentimes, you don’t have to throw away what you’re already doing. Oftentimes, you already have nuggets of really important things that you’ve built over time. And all you need to do is to work with somebody who sees things just a little differently and can tweak them with you. And I think that partnership with Elizabeth and Makita was really critical for me. They were able to take things that I already had, and show me a different way to just modify what I was already doing. That messaging would go out to every student after that time point. When you see an effect that size, it almost becomes ethical to in fact, it is ethical, in my opinion that you don’t preclude any student from experiencing it. So that should be the way that everyone should be communicating with their students, in my opinion. And you want to make sure that it’s authentic as well, because I started with a set of materials that I had already written and sort of came from my heart, my own experience, that made the authenticity, I think, real whereas if I would have tried to have said something to students that I really truly didn’t believe about their abilities, I think they would have sniffed that out in a heartbeat. And we probably wouldn’t have seen the effect that we did.

Rebecca: It’s amazing what a little bit of intention can do in the design of something, especially because we often don’t realize how big of an impact a simple communication might have. We almost think of it as like a throw away, or a quick thing you might be doing, but you add a little intention to it, and it starts having a really different impact.

Elizabeth: Yeah, I think you mentioned earlier that students might not read the email, you might think that they’re not going to engage with it. And we also thought that, [LAUGHTER] so we think that in designing this intervention, we were really drawing on a lot of the intervention research that had been done previously. So research on wise interventions, mindset theory, all of that was integrated into the design of the message itself. So we started with Bill’s original emails that he was sending out to students and the messages that he had already curated for his class, and timing it in a time in the class in which students are receptive to that information, I think, is also really key. So we don’t know this empirically. But I would guess that if we were to send that email a couple of weeks before the exam, it would not have the same kind of impact. We send it directly when that performance information was posted online. And that’s when students are questioning themselves. They’re saying, “What does this score mean? Does this mean that I’m not good at this? Does this mean that I can continue? Does this mean that I’m smart or not smart? …and that uncertainty, that is the right time to infuse your message as an instructor to counteract those uncertain messages about ability. So I think the timing really matters. I think authenticity, like Bill said, really matters. Makita and I have been working with other institutions to implement a similar type of intervention with instructors. And before we did that Makita did a bunch of focus groups with the students at these institutions and showed them the messages that we used in Bill’s class, and we ended up changing them quite dramatically. The students gave us feedback like “nobody here would ever say something like that,” or “here’s what you need to say to reach students here in my class,” or “here’s what I will be receptive to.” So students in different contexts are going to respond to different messages. And so you really do want it to fit your own class, how you would speak, how you would talk to your students, the needs of the students in that moment might be different from the class they’re in. So it really is not a copy and paste type of situation where you were gonna copy what we did here and put it in your class. I think you do need to kind of go through the process of generating it for yourself and your own students.

Rebecca: I could imagine there would be an element of needing to evolve it over time, too, as your student population changes, and as you change as a teacher.

Elizabeth: Yeah, absolutely.

John: One of the things that I think is particularly nice about this study is that we see higher DFW rates by first-gen students in STEM classes in particular, and so many students end up leaving STEM fields that they intended to major in in their first couple of years. And that’s where the rate of return to education is the highest. And in general, first-gen students come from lower socioeconomic groups and lower family income groups, and if we do want to see more equity in outcomes, getting more first-gen students into the STEM fields could do a lot to help reduce some of those equity gaps in terms of overall outcomes.

Elizabeth: You mentioned the long term nature of these effects. I think it’s a great question. I think it’s something that we’re currently looking at in a project that we’re doing that’s funded by the National Science Foundation, where we’re looking at how these interventions in an introductory course can change career trajectories, that when you get a B instead of a C in a major introductory course, that’s gonna mean something different to that student, that’s going to set them on a path of following that next course sequence, that next step. So that’s the question that we’re asking and the research that we’re doing now, and hopefully, we’ll have some answers in a few years.

John: Peter Arcidiacono at Duke had done a study on the decision to stay in STEM fields. And he did look at some of these questions in terms of the effects on major choice based on the grades that were received. We can share a link to that study in the show notes. In the IRB proposal, did you include the possibility of following up these students in terms of their outcomes a few years later, in terms of retention rates for the students?

Elizabeth: That’s a great question. I believe in terms of IRB, we had students consent to their academic records. So we should be able to go back and look at transcripts from the students who participated in the class. We could look at course taking, we could look at graduation rates, we could look at major. If these students were freshmen/sophomores in 2021, then we’re probably around the time where they would be finishing up so maybe in the next year we could.

Bill:I was just thinking the same. Now’s the perfect time to go back and take a look, because we’ve had enough time since the intervention to see what’s happened. And actually I was gonna go back to one of your earlier points. You were talking about statistics earlier. Basically, this is really the only way that we’re going to increase the number of STEM students going forward. If first-generation students at WSU are 40% of our population today, that means if we don’t improve their outcomes, then we’re going to continue to see the declines in STEM outcomes and STEM graduates that we are trying so desperately to fight against, and also, we’re not going to have the representation in our STEM graduates of the population of the US that we want to see.

Rebecca: I think in addition to the growth-mindset piece, I’m sure that the authentic nature of the messages also just showed deep care. And that can be really important for first-generation students who need that ally or advocate for them in their college journey. So I can imagine those messages coming from Bill in Bill’s voice that make them feel like “Yeah, you can do this, I care about you,” from that perspective is also really important.

John: And there was a study done by a couple of economists, entitled “My Professor Cares,” where they did follow up students four years later, and they did find a significant effect in retention rates four years later by a similar type of intervention, I don’t think it was quite as much growth-mindset focused, but just sending a signal that the professor cared about the students outcomes, a very simple message, made a significant difference in student’s grades as well as longer term outcomes. So it’s nice to see more studies of this, especially focused on issues such as growth mindset.

Elizabeth: Yeah.

Rebecca: So we always wrap up by asking: What’s next?.

Elizabeth: So Makita and I and our lab have been working for the past couple years on following up on these data in different institutions. We used this study as pilot data in a grant application. And now we’re sort of scaling up to other institutions. We have a project with a really large HBCU, a Hispanic-Serving Institution, WSU is continuing to be involved as well, an Asian and Pacific Islander Serving Institution to where we have a number of instructors who vary in different characteristics, different class compositions, and we’re looking at these instructor messages in an experiment in these different contexts to see what matters. Does it help students develop their own growth mindset? Does it change their behavior? Does it work in some contexts, but not others? Do the instructor characteristics matter? So far, in the past two years, we’ve collected data from around 10,000 students who have been involved in this intervention project, and we’re currently in data analysis phase. So stay tuned for future studies and papers that come out of that data set.

John: Makita?

Makita: Well, what’s next for me is Elizabeth and I are actually going to meet right after this to do a debrief and to discuss some of the analyses that I just ran on a study that will be involved in my dissertation.

Rebecca: And how about you, Bill?

Bill:For me, I think about: “How do I proselytize about this work?” Because I think about what little changes make a big difference in an institution. We talk a lot about student retention, we talk a lot about student outcomes, we talk about creating the global citizenry that we want to see emerge from higher education. And the one experience that every student has in common, whether they are a first-generation a continuing-gen, or from some other background, is they have to take classes, and they have to be in classroom environments every day of the week, for the most part, over a course of four years. And I think about what would it look like to have interventions like this, that have evidence that back them, that students get inoculated with repeatedly over multiple time points with different voices, and that faculty engagement in the process of helping our students become better, to find success and to move forward in their careers? To me, that’s the magic of this. I’ve been talking to other people about it and I think about what would a curriculum look like that had small things like this? And how could it transform what we do? And I think that’s a vision that I can buy into, and I hope others are gonna come along for that ride.

Rebecca: I think the scope of the interventions that you’re talking about, that have this kind of impact, are really easy to buy into. Because usually you’re thinking about some drastic, giant investment of time, is what most people are thinking about. And that often is what prevents action from happening. So I’m excited, excited to share this and to hear what comes of your bigger study.

John: And one thing we appreciate is when we see things like this posted on social media so we can hear about it [LAUGHTER] and we can share it out a little bit further. [LAUGHTER] And I hope you’re presenting this at various conferences to reach a larger audience.

Elizabeth: Yeah, we will be presenting some of this work at SABER this summer. So that is the Society for the Advancement of Biology education research, and we’ve presented it in some other psychology conferences throughout the past year as well.

John: Excellent.

Bill:And I gave a plenary talk at the summer educational meeting for the ASBMB, American Society for Biochemistry and Molecular Biology. So I gave a little preview of what we had found. And that generated a lot of discussion and excitement, actually people asking for additional resources and ideas and things. So I think there’s a receptive audience out there.

Rebecca: That’s wonderful.

John: Well, thank you for joining us. It’s always great talking to you and it’s great to hear about this newest study and to meet you, Bill. We’re looking forward to hearing more about how things go in the future.

Elizabeth: Thank you so much. Thanks for having us.

Rebecca: I hope you’ll reach out when your next study is done. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

347. CATs and AI

Classroom assessment techniques, initially developed at a time when chalk-and-talk instruction was the norm, helped to shift the focus from teacher-centered to learner-centered instruction. In this episode, Todd Zakrajsek joins us to discuss how generative AI can enhance these techniques by providing more immediate feedback.

Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published five books (so far) in the past five years. His most recent book is a 3rd edition of Classroom Assessment Techniques: A Handbook for College Teachers, with Thomas A. Angelo.

Show Notes

Transcript

John: Classroom assessment techniques, initially developed at a time when chalk-and-talk instruction was the norm, helped to shift the focus from teacher-centered to learner-centered instruction. In this episode, we explore how generative AI can enhance these techniques by providing more immediate feedback.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Todd Zakrajsek. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published five books (so far) in the past five years. His most recent book is a 3rd edition of Classroom Assessment Techniques: A Handbook for College Teachers, with Thomas A. Angelo. Welcome back, Todd.

Todd: Thank you, Rebecca.

John: It’s nice talking to you again. It’s been a while. I think the last time was last summer.

Todd: Really? It seems like yesterday. Time goes by. It was last summer. You came to my office and recorded.

John: That’s right.

Todd: It was fun. You left me a really nice set of tea. Wooh.

John: And a mug.

Todd: Yes, I have a mug for the show. [LAUGHTER]

John: Speaking of teas, [LAUGHTER] today’s teas are:… Todd, are you drinking tea?

Todd: Every time I’ve been on this program I’ve been drinking tea and I’ve had some pretty exotic brands and blends but, I’m sorry, today I have water, the lightest of all teas.

Rebecca: It’s a very light tea. I have English Tea Time tea.

John: I have Lady Grey today.

Rebecca: That’s a good choice.

Todd: Alright, I think I came in third place.

John: We’ve invited you here today to discuss classroom assessment techniques. The first two editions to this book were written by Thomas de Angelo and K. Patricia Cross. In last week’s podcast, Tom joined us to discuss the origin of this book, where you joined as a contributor for the current third edition of this classic book. What was your reaction when you were first asked to join in this project?

Todd: Well, this is actually kind of a funny one, because it goes back to Linda Nilson and Teaching at its Best. And Linda was retiring, and the publisher, Jossey-Bass wanted to do a new edition of Teaching at its Best, and Linda says “Why don’t you see if you get Todd to do it and I will work with him on it and do it.” And we did it. And we got the book done a little bit early and on schedule, and it came out very, very well. So there’s a little plug for that book there. And it must have been about two months later, I had called them and said: “If you find another project like this, I love working with people like this.” And a couple of months later, I got a phone call and they said “We have another project we’d like to potentially have you work on” and I said: “Great, what is it?”… and they said: “Have you ever heard of the book Classroom Assessment Techniques?” And I held it together, and I said, “Well, yeah, yes, I have.” And they said, “Well, we thought it’d be helpful given what you do and what Tom does and everything, is maybe the two of you could work together on this book.” And so he talked a little bit about what there was and everything, and I hung up the phone and did a little happy dance because this book was one of the most influential books in my entire career. But the way the universe works is I ended up doing this one because of the work with Linda Nilson’s. And so that’s how I got started on this.

Rebecca: That’s amazing.

John: And, I agree, that is an excellent book as well. So you’ve got two classic books that you’ve worked on recently.

Todd: It’s really surreal, you know, a first-generation college student and every once awhile is kind of like I just can’t believe this is happening. But it was a really good project.

Rebecca: When did you first start using classroom assessment techniques in your classes? You mentioned that this book was influential.

Todd: I taught my very first course in 1987-1988. This book came out, second edition, 1993. And I was using it pretty much the moment it came out. Somebody had pointed it out to me and I didn’t know anything about classroom assessment techniques, and then I started reading through it and picking some out. I thought it was genius. Much like periodically, what you’ll do is you look at something and think this is ridiculous that nobody started doing this before. In some respects, it’s not that hard, right? You ask people questions, and you find out if they’re learning. It just took Tom and Pat to come along and say, “Why don’t we come up with strategies to find out if they’re actually learning.” So I think it was really influential for me because it kind of struck that chord with me that I always believed we should be finding out how much students were learning instead of focusing on teaching. But that’s not what was going on at the time, everybody was focused on teaching-centered educational practices.

John: What do you find most valuable about these classroom assessment techniques?

Todd: It’s interesting, because there’s a couple different levels. The one that’s really valuable, of course, is you find out if students are learning. Secondarily, you can find out if you’re teaching well, and you can change your teaching practices based on what they’re learning and based on what’s going on. But I’ll tell you the real reason I think this is really, really valuable is it addresses equity issues very, very cleanly. We walk into classrooms with all kinds of preconceived notions and stereotypes and implicit biases. And it’s really, really easy to think that a student’s struggling for a certain reason, unless you ask them. And so, for several of these strategies, I think it’s great because I’ve had students that have been practically sleeping through every class and then I started doing the classroom assessment techniques and he had some of the best responses, this one student did. And I would just totally blown away that this person was just processing it to such a level and I had a preconceived notion based on where the person sat and how they dressed and how they acted in class. I didn’t think they knew anything or they didn’t think they were learning and they really were. So I think it’s huge for equity and overcoming some of the biases we have in our classrooms.

Rebecca: That’s a really good example to underscore the importance of that. We had the opportunity to ask Tom what his favorite classroom assessment techniques are. So I’m really looking forward to asking you the same question: what are your favorites?

Todd: I’m one who says over and over again, in education, that we should always be careful about absolutes. Like, there’s never a strategy that always works. There’s never a type of student that’s always the best. Any situation can change. So there’s like 55 CATs in there. And I listened to the program with Tom and I heard him say that he didn’t have his favorites. And then he quickly picked out his favorites, and I’m sure it’s the same with his daughters. [LAUGHTER] So I have three girls, you know, they changed the favorites, I would definitely go with the standards of the muddiest point minute paper, those two are the big ones. Everybody’s done a minute paper, probably at some point. The muddiest point I like, which was the adaptation. And if you haven’t heard this show with Tom, make sure you listen to that, because he explains that very well. I think I like exam wrappers where you go in and kind of have the students talk about the exam experience. But for me, there’s several CATs that are based specifically on assessments. And it’s the thing that we almost never ask students for, and it’s the thing they can give us the best feedback on. So it’s like, when you have group activities, it’s an assessment on how well the students perceived that the activity worked for them. So instead of me just saying, we did a great group project today, you ask the students: “Was this a good group project?” There’s assessments in there about learning interests, what types of things you’re interested in, your study strategies, exam strategy. So you can ask students those things. And I think those make great classroom assessment techniques, to find out the processes that you’re doing, and how the students are receiving those.

Rebecca: Those are good opportunities to respond in the moment in your class, but also probably plan in the future too, to know what to do next time around, perhaps.

Todd: Yeah, I think specifically for exams. And one of the favorite questions I’ve always had on exams, because the students would get to a point where, and I got this, because of one of the assessment techniques I’d use early on, is students always were frustrated that there was something on the test that they studied really, really hard for, and it wasn’t on there. The problem was, it wasn’t always the same. So the students all had different things they studied. So because of collecting that information, and finding out from the students as they were frustrated, I started adding a new question at the end of every exam worth the same as some of the other questions there, was: “Please describe in detail something that you study that I didn’t ask you about on this test.” And students immediately felt better about the exams, they were happier with the test, because they didn’t spend four hours memorizing something that they perceived wasn’t valued on the test, even though the real reason for all of this stuff is life, not the test. But again, that was one from just asking the students: “What do you like and not like about the test?”

John: That’s a really great application of a technique that I hadn’t thought about, I can see how students would feel that their voices are being heard with that type of thing.

Rebecca: …or that their efforts were validated.

Todd: They got their points for their studying, and it just really seriously changed the affect of the class.

John: In the last podcast we did with you last summer, we talked a little bit about how teaching has changed from the time when you and I both started teaching, and this book was written back around the time when you and I were both relatively new at this. What role do you think this book has played in that evolution of instructional practices?

Todd: Well, first of all, I do want to acknowledge and help people just really understand the difference between 1993 and 2023. Some of the things we took out of the book was “display your results on an overhead projector,” there was no mention of the internet anywhere in that book, because the internet wasn’t here yet. Just stop and think about that for a second. So it was a very different time. A lot of heavy lectures. I mean, a lot of times people were lecturing, there were other activities, or other strategies, there was group work and role plays and those types of things. But primarily people are lecturing, because that’s just what people did, primarily. And so the role that this played, I would say, I don’t know if I’ve ever said this before. I think this book had a bigger influence on higher education than any other single book that’s out that I know of. There are some really, really good books out there. But this fundamentally changed the game. Because what it did was it started asking students about their learning, which means it started to really shift the focus from teaching to learning. This did happen to come along, by the way, if you really look at dates, the whole concept of really pushing active learning in higher education, at least in the United States, really took off in the mid- and late-1990s. So 1996 to 1998, we started seeing people like Richard Hake, do some huge studies about this. American Association of Higher Education was changing their themes from taking teaching seriously to taking learning seriously, all of those things happened about three to five years after this book came out. So I think this was the impetus for a huge, huge aspect of the whole push for moving from teaching to learning. I think it had that big of an impact.

Rebecca: In recent months ChatGPT has hit the news everywhere. We’ve heard all kinds of reports in higher education journals and things. accreditation organizations are talking about AI. And many faculty members are also talking about potential harms of AI or other see benefits. What do you think the impact of AI tools will be on higher education practices? …speaking of evolution… [LAUGHTER]

Todd: I don’t have the statistic right in front of me, but there was one I did a presentation just recently on how long it took to get to a million users. And they were talking about some things like Netflix and other things of taking two and a half years, and then six months and stuff. And I believe ChatGPT was five days. So it’s pretty incredible. So Rebecca, I love the question. I think this is a huge issue. The two biggest things that I heard out of the gate was: number one is how are we going to ever have students do their own work again? It was all academic integrity all over the place. And I couldn’t help but talk to a couple of people who were really freaking out about it and I said, “You should really read some of the research about academic integrity already and the proportion of students who are doing things that they shouldn’t be doing, so to speak, there.” And so that was it… academic integrity. And number two was like, “What kind of assignments should we do now? How do we structure assignments to be meaningful?” And I find this fascinating, because they weren’t talking about the learning process. And I still, to this very moment, I’m just starting to hear it now, but I have not heard much of anything about how can we use generative AI to influence the learning process in a positive way. And so I think that’s the huge thing we need to be looking at. So I guess the people who do know me, know me as the individual who writes a lot about helping students to learn and teaching tips in the classrooms kind of things. I think from a cognitive psychologist’s point of view, we can look at some of the foundational things like: we know that repetition is really good, we know that the process of explaining something really helps because it’s a chance for you to practice at retrieving information. We know spaced recall is really good. And there’s all these things out there that can kind of do that. But I’ll tell you, and I’m coming right back to the CATs, but ChatGPT and some of the other generative AI has some phenomenal opportunities. We know that teaching this is good. So I opened up by ChatGPT 4 the other day, and I was practicing and I typed in: I need to learn about metacognition, would it be okay if I taught you metacognition, and then you tell me how good of a teacher I am. And the chatGPT says, “Yeah, this will be fun. So let’s get started.” And I said, “Okay,” and so I explained metacognition. And ChatGPT asked me some questions like, “Well, what does this mean? And what does that mean? …and I acted just like I was teaching somebody. And then when I was all done, I said, “If I were a teacher, right now, how would you grade me?” And I did not get a good grade [LAUGHTER] which was kind of disconcerting there. But I said, “Why did I not get a good grade on this? And then it explained a section that I had completely forgotten, because I was doing this off the top of my head to try it. So I went back and explained that. And then ChatGPT says, “That’s much better.” I think that’s phenomenal. Another one, I will not get into details, but I typed in, or I gave it the prompt… it’s all about the prompts folks… “That I just learned in my introductory psychology class at a medium-sized, midwestern university. I’m a C-level student who’s worked really hard, and what I just learned were about persuasive techniques. Could we pretend we’re on a car lot, and I’m a salesperson, and you’re going to buy a new car, and you think you want to buy it today, but you’re not going to rush into anything.” And ChatGPT says, “This sounds like fun. Tell me about your best selling model.” [LAUGHTER] And I proceeded to explain all these things. And the scariest one is I said, “This Ford van right here has excellent ratings for safety.” And it said, “That’s particularly important for someone with a loved one and two furry friends.” So the learning possibilities are all over the place for this thing. And so for classroom assessment techniques, this is great, because we can actually teach students some strategies, how in the classroom, how to assess whether they’re learning. We can replicate these and have them do them with AI. So we could use AI to do things we couldn’t do before. But we can also do it to model things for students that they can do on their own. So I think there’s some really, really interesting possibilities coming along.

John: And one interesting one is if you ask ChatGPT for your biography, you may very well turn out to be a car dealer in the Midwest.

Todd: You know what, I’m just gonna jump in there and say that is totally true when ChatGPT was about six months old, because I did it. I said, “Write a letter of recommendation for Todd Zakrajsek… and you probably saw this, why you even mentioned it… and I think it had 19 things I counted in there and this term of hallucinations, but 17 of the 19 things were wrong, including where I went to school where I worked and everything else. That was when ChatGPT was a baby. I asked it the same thing just recently, and it was scary, how close it just nailed everything. But yeah, it was for a while.

John: It’s improved quite a bit, but one of the nice things about it is our students come in with really diverse backgrounds and we can provide supportive materials for students when they have some gaps in their prior learning. But, ChatGPT offers the possibility of having something that’s completely personalized to their needs, so that they can ask questions that are specific to the issues that they’re having trouble with, and not what we might have guessed they’d be having trouble with. So, it’s something I’ve encouraged my students to use in my intro economics course, and they appreciated the fact that they were encouraged to use it,

Todd: I think that individualized instruction is going to take on a whole new level. And so I think that’s going to go crazy. But I will say this, and I think it’s extremely important. I’ve been in higher education for 40 years, I have never seen a bigger possibility for an equity gap to be the widest chasm it could ever be. I think we got equity problems that are down the road. And if we don’t pay attention, it’s going to be awful, because I believe that generative AI is going to be a huge learning aid, and it’s going to help us with lots of things, for the people who have it available to them, they will skyrocket and be able to do stuff; the people who don’t have it available… reliable internet, a good computer, or some kind of a device, a safe place to sleep at night… those individuals are not going to be on that tract and they’re gonna get separated. Right now, there’s a couple of grade differences between people who are privileged and not in some classes. That could widen so much. So I think, everybody, we just got to really be careful about how to make sure that this was as equitable as possible for everybody.

Rebecca: Higher ed leadership at various colleges and universities are certainly dancing around the idea of AI and what kinds of policies and procedures to put into place. What would you advocate for? What would you encourage faculty members to advocate for to really make sure that equity is addressed?

Todd: I think one of the very first things we need to do is teach students how to use it properly. Because one thing we know about learning is, the more you know about something, the easier it is to learn something related to that topic, whatever the thing is, and what that means is that learning is not linear, it’s curvilinear. The more you know, the easier it is to learn something, which means you’re going to know more quickly, and as you know more quickly, you’re going to be able to learn more quickly. And this is why people struggle at the beginning with things and then all of a sudden they take off. So if some students know how to use the systems, and some don’t, that’s where we’re going to run into some huge issues. So I think one of the things we can do is teach students how to use it to help you, and when to be careful that it’s not going to help you. The movie Wall-E is a great thing to be keeping in mind. If you haven’t seen the movie Wall-E, it’s a plug for the movie Wall-E, it came out a long time ago, but that concept of the humans that had everything done for them ended up becoming just blobs that laid in chairs all the time, and I think our brains basically do that. If students use AI to do their work, they’re not going to develop critical thinking skills. If students use AI to help them to practice at retrieval and spaced out practice and do those things, they could become very good. So I think schools need to be careful about helping students to understand how this can really help them. That’s the first thing. Then there’s the obvious, helping people to have access to technology, we could have labs open, we could have laptops that we can loan to people, reliable internet, we could have rooms that we keep open 24 hours a day, keep them 24/7 there. Some libraries during exam time, they don’t close for like four or five days. We could have rooms in the library that people can come and have safe spaces to study and work, reliable transportation is going to be an issue, we’re just gonna have to work out how do we address those issues. It’s gonna be a challenge, but we need to be thinking about it.

John: The use of classroom assessment techniques has been growing steadily since they were first introduced. And each time there’s been new educational technology, it seems to have led to an increased use of those. During that time, we’ve seen the introduction of relatively low-cost computers, computer networks, the internet, mobile computers, and so forth. How do you think the availability of generative AI will affect the use and value of classroom assessment techniques.

Todd: I think there’s a couple things that’s going to really, really change. I think for classroom assessment techniques, number one is we’re going to be able to individualize a little bit more. So we can tailor… we don’t have to ask the same question of all the students, we don’t have to look for all the same type of responses. So we can think about it a little more creatively in how we can use it, almost like an individualized instruction toward classroom assessment. Probably the biggest thing, though, overall, is it’s going to allow us to just process data in numbers and levels we’ve never seen before, and particularly free responses. In the past, we’ve often ask closed-ended questions just because if you’ve got a class with 400 students, you can say: “On a scale of one to five, to what extent did today’s lecture help you to understand something?” And the students can pick a number, we could drop that into an excel sheet, or anything, and come up with a number very quickly. We can now say on a muddiest point, “What are you still struggling with?” Take 400 responses, dump them into a generative AI program and have it spit out five things within 30 seconds:” Here are the five things your students are struggling with.” So I think it’s going to allow us to do more qualitative types of things very quickly. And I totally get that this is not hardcore qualitative research with good analysis of the data. But for what I need in the classroom, I think we’re gonna be able to get those responses very quickly and in real time. So what that means is, we’re actually going to be able to ask something like a muddiest point at halfway through the class, and then I could have the students do a quick think-pair-share, while I analyze the information and a matter of three minutes later, they come out of the think-pair-share, to talk a little bit about what they talked about. And then I could say, I see that you’re still struggling with these concepts, so let’s revisit these things before they ever leave the room. That’s never been possible before.

John: When I’ve been teaching classes of two to 400 students, I’ve used the free response option and word cloud. But the word cloud is just highlighting individual words, the ability to do analysis in more detail is going to be incredible. And I think most providers of response software are working on introducing AI, and some of it is expected to be available fairly soon.

Todd: Oh, there’s some great stuff come check GPT and some terribly scary stuff at the same time. By the way before I forget this, because I have to tell you this because I have ADD, that’s one of my favorite phrases.ChatGPT 5,, one of the things I read about that’s going to be coming up fairly soon is the ability for it to launch his own AI’s as needed. That’s the one that we’ve been waiting for, and thinking, “Hmmm, that’ll be interesting.” But yeah, I think that the ability to read through and look through the information is just going to be a game changer. And online synchronous/asynchronous, face-to-face in all environments. We could look at stuff in asynchronous environments in ways we haven’t before as well. I could look at different ways that students are responding, I could ask classroom assessment techniques. In fact, they’re in the book. We have several of them… ways to use CATs in asynchronous environments, and AI is changing how we’re doing it.

Rebecca: Can you give an example of how that’s happening in asynchronous environments?

Todd: In terms of asynchronous environments, everybody likes to go to discussion boards, but quite frankly, discussion boards are boring. We could have students generate something like the script for a commercial. And after they do that, I could have the CAT that comes in to say, “To what extent was this helpful in the learning process for you?” And the student, as they’re developing this thing and as submitting it they could also submit, right along with it, their CAT that comes with it, and then I could be reading the CATs as they come in one by one right behind each one of them. And so the concept, there, would be kind of a real time CAT analysis that’s not waiting until there’s a whole group and then looking at the group. Because typically, when we think about these CATs, it’s like, the 400 students fill out a muddiest point and we analyze the 400 students. We don’t need to do that. In an asynchronous environment, we can have these CATs coming in and we can be analyzing them as they come in. And we could even add them to the previous ones, to group them if we wanted to. But the bigger one is, I can find out how the students are doing as they’re doing. So that’s probably one way to look at it.

Rebecca: One of the things that you mentioned in an in-person or a synchronous setting was being able to analyze results mid class. And that’s in part because a lot of our students have mobile devices and technology that they bring with them that they haven’t had before. Do you see other ways that these mobile devices are also changing the way instructors might implement CATs?

Todd: The way you just mentioned, though, I can’t think of many other than the concept of as an information gathering device. I will say, just because I happened to read this study just a little bit ago here, the devices are very interesting, in a sense that we’ve seen high schoolers tend to have the same amount of time they spent with their peers was very consistent until about 2012. And in 2012, the numbers started dropping off pretty quickly. And then COVID dropped it even faster, but the slope was already there. And the point here is that people are turning their attention to devices so much that they’re not talking to each other exhibited by when you see two people at a restaurant having dinner, and each one of them are on their device. There’s all kinds of psychology about how those things happen. But the overarching thing that’s the issue is that we’re having students at levels we’ve never seen before disengaged in the classroom, even when we’re trying to do CATs and engage. And part of that is because of the devices. So one of the things that’s kind of interesting about this is when you say: “Okay, everybody gets your devices out, and now we’re going to use them for a CAT.” And then we stick with that and how we use that data makes a big difference of whether they stick with it or go back to what they were doing. Strategically, it’s really helpful to use those devices to engage the students.

John: One concern with generative AI is many of the types of assessments we’ve used before can be answered very nicely by generative AI. What might we do to reduce the likelihood that students will use generative AI as a substitute for learning?

Todd: I think it’s really, really important to talk to students about the long-term implications. I know this is not going to be for everybody. I totally get that. But one thing I’d say to students is: “I don’t understand why anybody would run, why would anybody go jogging? You can just get in your car and get there faster.” So if you kind of pitch that to them, that idea is for most of the students, if there are any kind of health-related fields, or if they want to have a cardiac system later in life, exercise is just one of the most important things you can do for your body, by the way, in terms of that getting 150 minutes per week, and just higher respiration. If you take that away, it is bad for the human body. And so the response is the same thing is true of this. Just talking to the students. If you use generative AI to answer your homework problems, to develop the quizzes, to write a poem, and I’ve done all of these in workshops, where I’ve said to people, I can do this real quickly. I don’t care who’s in this room, I can ask you to write a short story. And I will crush you with generative AI. The problem is that, if I do that, I don’t learn how to think for myself. And so I think the biggest thing we can do for students, number one is to build community in the classroom, to number two, to tell them what this all means, and then number three, ask them, ask them, how it’s helping them, ask them how it’s hurting them, how they believe these things are working. And those are classroom assessment techniques, we can use these assessment techniques to find out to the extent that they’re doing this. While I’m on my little diatribe, we will never be at this spot again in our lives. We have just developed something that can totally short circuit cognitive processing and critical thinking. But we did it with people who have critical thinking skills and cognitive processing. So we developed something because earlier in our lives, we had to learn in the ways of joggers, we had to develop our systems. And now we can talk about the automobiles that would move as faster. The students coming along are going to step right into those automobiles, right into the AI, and if they’re never told that it’s important to go out jogging, they’re not going to develop the systems that are going to be needed later. And so again, just think about this for a second, we’ve developed a system using a system that may disappear if we’re not careful. And so I think that’s where we can change it. So it boils down to community, why we’re using it, and then assessment questions of how it can be used, and where it’s good and bad.

Rebecca: I think students really enjoy having conversations about AI and exploring how it’s useful and not. I have had activities in my class where we intentionally used AI to see what it was like and when it would be useful and also analyze where maybe they tried to use it and it was totally not helpful, and why it wasn’t. And they really appreciated those kinds of conversations and learned a lot from those.

Todd: I think the students do. And we could pick different spots in history. You go back to Socrates and his whole belief that if you write things down, it weakens the mind. It makes sense. If you have to learn it, you’re going to be much stronger than if you write it down. But I can’t imagine right now teaching without students writing things down. And Samuel Johnson came along several 100 years ago and said, “There’s ready availability of books. With books all around, why would we really need to teach this stuff, they can just go get a book.” I can’t imagine teaching without books right now. So these things that everybody got scared about, or thought that just going to change just became integrated. And when the internet came about, we talked about this a while back, in the sense that many of us were teaching before the internet actually showed up. But when it showed up the people who were teaching were freaked out. “How am I supposed to teach when the students can go and get anything they want off their computer?” And now I can’t imagine teaching without the internet. I believe five years from now people are going to say I can’t imagine teaching without writing things down, without using some kinds of print format, without using the internet, and without using generative AI. I don’t even know how I would teach without it. What that’s going to need is the same stuff we’ve done in the past, that’s how do you teach well with the Internet? How do you teach well with generative AI?

John: It’s an exciting time to be in the midst of all these changes. And it’s going to be interesting to see how we all answer those questions as we move forward.

Todd: It is, but before we wrap up, because I can tell that tone, [LAUGHTER] I want to put a general call out that there are several people who are saying: “I wish I could retire right now” or “This is a great time to retire.” The statement I really want to make is this is a hideous time to retire if you’re really good at what you’re doing. Because we have never needed humans with really good critical thinking skills as much as we do now, maybe we will later, but to date I don’t believe we ever have. And for some of the people who are saying “I just don’t want to teach anymore. This is awful…” we might need you more than we’ve ever needed you, so I don’t like this concept of the mass exodus of certain people.

Rebecca: It’s an important point.

John: One other thing I will add is Paul Samuelson once said, in describing the way in which models of the economy evolved, that “funeral by funeral, the science makes progress.” So there is the counter argument there too, that people who are very tied into the old ways of teaching do need to either adapt or perhaps they’ll be replaced by people who are more willing to try new alternatives.

Todd: Do I smell a little Kuhn in there? The revolution of science. Ah yes.

Rebecca: And we always wrap up by asking: “What’s next?”

Todd: I’m trying to keep busy. I’m kind of on a roll right now. And I want to tell you, I’ve been writing books and the only reason I’m unhappy about ChatGPT and gen-AI come along is people are going to assume that I’m using them so that I can crank stuff out. But thus far, I have not used any of this generative AI to write any of these last books that I’ve done, these five books in the last five years or something, and enjoyed working with Tom on this one. But I got another one coming out in about four months. I just turned in the final pages for that And it’s the Essentials of the New Science of Learning: the power of learning in harmony with your brain. That book should be out in a couple of months. And so the what’s next after that, I am working on a book right now for helping with neurodivergent learners. We’ve joked around in the past, I’ve got ADHD is about as bad as it gets. If it were a competition, I could come in probably close to first place, and also on the autism spectrum just a little bit too. And when I started sitting down with some of my colleagues who also have ADHD and autism spectrum, it occurred to me, we don’t talk to students enough about these things. And I think the challenge of that is that the students believe that they’re alone, and they’re not sure they can do things. And I was talking to a student fairly recently that said, “I now believe I have a shot, here you are with a PhD, you’ve written books, if you’ve got ADHD, as bad as you claim you do, maybe I can do something.” And then that’s when I really really was getting serious about we need to help folks out. And the topic I’ve been playing around with a little bit in a couple of workshops is The Ones Too Often Left Behind as the title I’m using. And it’s the students that aren’t built for the system that we developed. And so I think that we need to treat some folks out there a little differently. And I think we can really build up the pool of intelligent folks by helping to teach the people who just learn in a different way.

Rebecca: It’s really important work to have models and put models out in front of students because they need to see themselves in whatever discipline, field, etc., that they want to pursue. I’m excited to hear more about that.

John: And when will these books be out? You mentioned the timeframe for one of them.

Todd: The Essentials for the New Science of Learning should be out in September. And then the other one, if everything goes well, is probably looking at a February date that it would probably be available.

Rebecca: Well, it sounds like we’ll be talking to you soon then, Todd.

Todd: We may be. When you got ADHD pretty bad, you just can’t really predict when that books gonna be. But yeah, I’d love to chat with you when it does finally emerge.

John: And what are you going to do next week?

Todd: You know what? You gave me a great idea. I think we should do a book on procrastinating people with ADHD. [LAUGHTER]

John: It’s always great talking to you. And we’re looking forward to each of these projects coming to fruition and we will be talking to you about each of them, we’re hoping.

Todd: I appreciate the opportunity to come and chat with you. I think you two do a phenomenal job with your interviews and the programs that you pull together on really good topics and I just am honored to be one of the guests.

John: Thank you.

Rebecca: Yeah, thanks. We always enjoy talking to you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC] </p

346. Classroom Assessment Techniques

Classroom assessment techniques can be used to shape instruction to the needs of our students. In this episode, Thomas A. Angelo joins us to discuss the origin of these techniques and evidence concerning their efficacy.

Tom is Clinical Professor Emeritus and Director Emeritus at the UNC Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill. In addition to teaching for 40 years, he has been long involved in professional development and has served as faculty member and Director of teaching, learning and assessment centers at UNC, LaTrobe University in Melbourne, Australia, Victoria University of Wellington, New Zealand, the University of Akron, and Boston College. Tom is best known for his work with K. Patricia Cross on Classroom Assessment Techniques: A Handbook for College Teachers, which was first published in 1988, with a second edition in 1993.

Show Notes

  • Cross, K. P., & Angelo, T. A. (1988). Classroom Assessment Techniques. A Handbook for Faculty. Jossey Bass Wiley.
  • Angelo, T. A., & Cross, K. P. (1993). 2nd. Ed. Classroom assessment techniques. Jossey Bass Wiley.
  • Angelo, T.A. and Zakrajsek, T. (2024). 3rd. Ed. Classroom Assessment Techniques: Formative Feedback Tools for College and University Teachers. Jossey Bass Wiley.
  • Schwartz, Charles (1991). “An Academic Adventure.” March 4. – A description of the origin of the minute paper by it’s originator.
  • Chizmar, J. F., & Ostrosky, A. L. (1998). The One-Minute Paper: Some Empirical Findings. Journal of Economic Education, 29(1), 3–10. (the article that John referenced as his first reference to this topic)
  • Mazur, E. (1997). Peer instruction: A user’s manual. Pearson
  • Mazur, E. (2014). “Confessions of a Converted Lecturer” – a recording of a presentation by Eric Mazur at SUNY Oswego, 5/19/14
  • Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
  • Pam Hook and Josie Roberts (2018). “The who, what, when, where, and why of SOLO taxonomy.”
  • Angelo, T.A. (1993). “A Teacher’s Dozen—Fourteen General Research-Based Principles for Improving Higher Learning.”
  • Pre-order for 3rd edition of Classroom Assessment Techniques: Formative Feedback Tools for College and University Teachers.

Transcript

John: Classroom assessment techniques can be used to shape instruction to the needs of our students. In this episode, we discuss the origin of these techniques and evidence concerning their efficacy.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Thomas A. Angelo. Tom is Clinical Professor Emeritus and Director Emeritus at the UNC Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill. In addition to teaching for 40 years, he has been long involved in professional development and has served as faculty member and Director of teaching, learning and assessment centers at UNC, LaTrobe University in Melbourne, Australia, Victoria University of Wellington, New Zealand, the University of Akron, and Boston College. Tom is best known for his work with K. Patricia Cross on Classroom Assessment Techniques: A Handbook for College Teachers, which was first published in 1988, with a second edition in 1993. Even if you’ve never heard of these books, you may have included some Classroom Assessment Techniques in your teaching toolkit. Welcome, Tom.

Tom: Good to be here.

John: We’re very happy to talk to you. Today’s teas are:… Tom, are you drinking tea by any chance?

Tom: No, I was drinking black coffee.

Rebecca: It’s one of our most popular forms of tea [LAUGHTER] as regular listeners know. Today have some English tea time, John. How about yourself?

John: …and I have an Earl Grey tea today.

Tom: Excellent.

Rebecca: We’ve invited you here today, Tom, to discuss the third edition of Classroom Assessment Techniques, to be published shortly. These techniques described in earlier editions of this book have been widely adopted by faculty in all disciplines at all different types of institutions. And a lot has changed in higher education since 1993. But this book is still the go to reference on classroom assessment techniques, which is pretty impressive [LAUGHTER] for a book that was published 30 years ago. Can you talk a little bit about the origin of this book?

Tom: Well, I think the origin of the book goes back to the fact that when I was a young teacher, I recognized pretty early that many of the things that I was doing didn’t work and didn’t work for my students, and that I had to accept that things that had worked very well for me as a student weren’t working for many of them. So I began to experiment with asking questions in a variety of different ways and getting students to jot down answers anonymously. And through that, I really learned more about teaching and learning, I think, than from anything else that I did. And one of the things that I learned was not to assume. So, classroom assessment techniques are things that I’m sure almost every teacher has done. But what we did in the 1980s. K. Patricia Cross and I, was to really formalize those. And we did that in order to create a resource for teachers so that they could find simple tools that would help them ask students the kinds of questions that would help students learn better, and teachers teach better, as they went along. So it really began at that time, and it was kind of an accidental start, in many ways. We were doing it as part of a big project, and what was then the assessment movement, and it just kind of took off.

John: Most of our listeners are probably quite familiar with classroom assessment techniques and have seen them at lots of workshops and have probably, as Rebecca noted before, used them in the past. Could you define what a classroom assessment technique is, for those listeners who are new to the concept?

Tom: Well, in a formal way, a classroom assessment technique is a formative assessment technique. It’s a way of gathering data to use to improve and inform practice, rather than to make judgments. So we have too, most of us, to give students tests and exams and assign papers, and grade their work, evaluate their work, for grades and marks. Formative assessment techniques, on the other hand, are tools that we use to gather data along the way, before those moments in which we grade them, or mark them, so that we can help students get ready and improve and succeed when they’re going to be graded and marked and so that we can, at the same time, find out where they are and how they’re doing in order to help them improve along the way. So they’re really ways of diagnosing, in a way, students’ learning.

Rebecca: One of the most popular classroom assessment techniques described in your book is the one minute paper. There have been many studies of the effectiveness of this technique and it’s been widely adopted in many disciplines. Why was this technique, in particular, been used so extensively?

Tom: Because it’s simple and easy to use, probably, and it takes almost no time. But seriously, the one minute paper, the minute paper was invented, to the extent these things are invented by Charles Schwartz, in the 1970s, at UC Berkeley, and he used it as both a way to take attendance (in his case, he was asking students to jot their names down), but more importantly, to find out whether students were learning from his physics lectures, in his huge physics classes, what he hoped they were learning. And that idea spread through UC Berkeley really fast. And so soon, there were dozens and hundreds of people using these two questions. And the two questions, famously, were: what are the two or three most important things you learned from today’s lecture, in his case, and what questions do you still have the end of the lecture? So students would jot these down on those strips of paper, remember, there was no internet and no computers, really, in classrooms in the 1970s. And then he would collect them, and read through a sample of the 1000 or so that he had every lecture. And from that sample, he would gather ideas about what students understood and didn’t from the lecture, and also what questions were most common. And then he would begin his next class meeting, by going over some of those points that students had had trouble with, or not understood, or missed them completely. Sometimes it happened that a 1000 students didn’t get one point that he thought was critical, and also answering one or two of the most important questions. And what he found was, he could do all of that in five minutes at the end of class, and about five minutes at the beginning of class. And I think it’s that efficiency that has made it so popular.

John: My first exposure to this was a study that was published in the Journal of Economic Education way back around the time of your last edition. I did try to find it, but I couldn’t find the citation for it, but what struck me was that the instructor had used it in one section of the course, had not used it in another section, and found that there was a very substantial difference in learning outcomes between the two groups, which he attributed to the one minute paper. And it’s been incredibly commonly used. But one of the things that really struck me is that at the time when you came out with this book, including the one-minute paper, and so many other classroom assessment techniques, most instruction at the time was the chalk and talk type where people were at a blackboard, maybe a whiteboard, if they were a little bit more advanced in the technology, where most assessment was done with high-stakes assessments. So I think your book was a fairly important factor in helping people shift to more formative assessments. Why have these techniques become so widely adopted?

Tom: Well, I think that one of the reasons why people have found classroom assessment techniques and other formative assessment techniques, there are other kinds, so useful and so important, I think there are several reasons. One is that the student body in the United States and across the world has become ever more diverse in the last 30 or 40 years. And it turns out that no matter who we are, as teachers, most of our students are going to be pretty different in terms of their experiences and their interests and their skills than we are. And that level of difference has probably grown over the years, at least our awareness of it has grown. So I think most teachers now realize that it’s very important to check in on students and see how they’re doing, before we get to the midterm or before [LAUGHTER] we get to the end of the course, and to check in regularly and not to assume that students are understanding our brilliantly clear presentations or the material that we give them, or the readings, or the problem sets, and to really check early on to see if it’s making sense to them, and what kind of sense it’s making to them. So I think that there’s much more realization of the diversity of human learning, and of our students, than there was 40 years ago or 30 years ago. And I also think that as teaching has evolved, especially as it’s become online, people have recognized that we need data. We can’t just depend on students volunteering that data, they often don’t know that they don’t understand. And that that data, used well, can help us and can help them.

Rebecca: I think one of the things that you’re talking about in terms of assumptions is reminding me of some of the conversations that we had early on in the pandemic, where instructors we’re talking about not being able to see the sea of faces to know whether or not students are learning because maybe students aren’t nodding along or providing some visual feedback, but classroom assessment techniques are really important because they allow opportunities to gather that data, which is probably far more informative than the head nodding.

Tom: I think there are maybe a couple of other things that are important about this that people discover. One is that by stopping to ask students to reflect on their learning and respond, you give them time to think. And you prompt them to think about their learning. And from my point of view, forever, the most important thing about doing this has really been to give students tools to become more aware, we could say more metacognitively aware, of their own learning, and to better take responsibility and control or at least direction over their own learning. So what I found is I’ve had many, many, many very bright students who had never been taught any kinds of techniques to think, or to ask themselves questions, or to monitor their own attention, and having those sorts of tools, those sort of metacognitive tools, I think empowers students to make decisions about their own learning, and also to improve their own learning in those areas where it matters most to them, which may not be my course.

John: One of the things you mentioned was that the one minute paper was originally developed in a class with 1000 or so students. And one of the nice things about many of these techniques is that they can scale that way. Now, there was a time when the book came out when this was all done on paper. But now we have so many electronic tools that we can use to do this. Instructors could put up a QR code with a link to a Google form or something where students can give their feedback within a minute or so or we could use polling techniques, and so forth. And many polling systems have built in one-minute questions or some other classroom assessment technique as a default strategy. So this impact has been pretty substantial. And it’s really nice for instructors, not just for students, having that time for reflection, but also for instructors to get some feedback on what students have learned. Because if you just call on students, you may happen to get the only student who understands the material responding to a question or you might get the only student who hasn’t understood the material. But with these formative assessments, you can get a good feel for where students are. And they also get some feel for where they are in a way that they might not if they would just passively sitting there listening to a lecture. So these techniques have, I think, become increasingly more common from the time of introduction. So these techniques have held up pretty well. But you now have a new edition of this book, a third edition of this book. Could you describe some of the changes in the new addition?

Tom: Well, obviously, and you’ve touched on this, technology has changed. And that’s had an impact on teaching, and an impact on learning. So one of the things that I did, in going back to do this book after so many years, was to take a look at first what was out there, what had people published on this. And there are literally about 3000 published articles of various kinds and various quality, about classroom assessment techniques. There were about 300 that I thought were of sufficient kind of quality and rigor to be able to use. And in the chapter on the minute paper, I refer to 150 articles and studied them very carefully. There are many, many books that refer to the classroom assessment techniques. And I also read [LAUGHTER] all of those references. So I took a few years to work through this material and new things were coming in all the time. So one thing that differs is that we have an evidence base across many, many disciplines for what seems to work and what seems to work well. And also much useful information on how to use these with various technologies. So how best to use formative assessment with online, fully online technologies on totally asynchronous classes. And so that’s been really valuable. And those are experiences that I couldn’t have all of those experiences. So it’s terrific to draw in those from other teachers. The other thing is people have invented new techniques or developed new techniques over time. And so in the third edition 25%, or about 15 of the 55 techniques, are ones that didn’t exist in the second edition. And that’s about one quarter of all the techniques in the new book. So one of the reasons why we have different techniques in the 3rd edition is that some of the ones in the second edition weren’t much written about, weren’t much commented on, and they seem not to be much used as far as I could tell. So I’ve picked up the ones that were most used and most written about, and most referenced so that people in different disciplines can find those techniques. And I’ve also found another 15 techniques to kind of fill those gaps and extend it a bit.

Rebecca: Your book offers many, many assessment techniques. I’m curious what your favorite few are that you’d like to introduce instructors to?

Tom: Well, just like I can’t say which of my daughters are my favorite, I do have favorites. I’ll be honest. I have always used the minute paper ever since I learned about it all those years ago, and I learned about it really when I was at Berkeley, in the mid 1980s. And I learned about it, as everybody else there did, kind of by chance, and thought, “Wow, why didn’t I do this? Why didn’t I think of this?” So the minute paper, the muddiest point, which is a variant of the minute paper, which is developed by a really famous and eminent statistician at Harvard, named Frederick Mosteller. And Fred Mosteller heard Pat Cross talk about this, when we were both at Harvard, and being the kind of person that he was he immediately experimented with it. What he found was that it was more useful for him, in his courses, to just ask one question, and that question for him was “What was the muddiest point In today’s class? What was least clear to you?” Many other people have used that as well and written about that. That’s another favorite. A third one is one that I’ve used a lot, being a statistics teacher. And that’s predict, observe, explain, which is often called POE. And that’s one that I’ve found useful in many situations. And I’ve had colleagues who’ve used that even in literature courses, and other courses where they’ll say to students, “let’s stop at this point in the book or the film, and predict what’s going to happen next, write that down, then we’ll observe it, and then we’ll try to explain why that either met our predictions or didn’t meet our prediction.” And while it’s obvious how people do that in science, and it’s part of the scientific method, it can be used in all kinds of circumstances. And it’s one that I’ve used with teachers who I’ve worked with, about classroom assessment and about formative assessment. I’ve asked them to predict what they thought would happen, then observe it, and explain what did happen and whether or not that met their expectations. And that’s been a very useful tool for me, I think, and the teachers I’ve worked with. So those are three good ones. I’ll give you one more, if that’s helpful. And that’s something that we call the directed paraphrase. So three of those four were in the first and second versions of the book, predict, observe, explain is in the third edition, but directed paraphrase simply asks the questions that are sometimes called journalistic questions, who, what, when, where, why, who did what to whom, when, where, why, and how? And it asks students to answer those questions about some phenomenon that they’re observing, or to answer them about an essay that they want to write, answer them about any kind of question. So giving students a framework of questions that they can work through, to make sure that they really understood either a book that they’ve been reading or an experiment that they’ve been doing, has been really valuable to my students. And I’ve said to them, “Look, this is something that you can adapt and take with you anywhere to use to make sure that you understand what’s going on fully. And to find out which part of it maybe you really can’t answer, and need to do some more work on.

John: In terms of the predict, observe ,and explain thing. I took a MOOC that was put together by Dan Ariely on behavioral economics. And one of the things he talked about was how he would often present the results of an experiment where the results were not what most people would have expected. But then he’d asked people for their reactions only after he summarized the results of the experiment. And they all nodded and said, “Well, that’s exactly what they would have thought would have happened.” So he started revising his presentations, where he asked people to make a prediction, and then revealed the result. And it got people much more engaged. And the same type of thing is done by Eric Mazur with his polling technique, where he asked people to answer a question where the question is not obvious and where there’s going to be some disputes. And then once they’ve committed to an answer, they work with other people and he goes through it again, with a little bit of peer instruction. But the level of engagement when students have made some type of a prediction of what they think the answer is, and then they have an explanation of what the answer is either given by the instructor or by other students, the level of engagement and interest just seems to go up fairly dramatically. So I can see how that would be particularly effective.

Tom: Yes, and I’ve learned a lot from the work of Eric Mazur, as have many others. And I think that that’s a critical thing. Because one of the things that we need to help students with, and help ourselves with, is managing our inbuilt biases and that issue of when you hear something new, feeling that you’ve always known it, even though you haven’t, is a cognitive bias that’s common, I think, to all human beings, myself included. So doing something like that: predict, observe, explain routine gives us a chance to really check and see if we know the answer. And then to find the answer, or discover an answer. And then to realize, because we have a document, we have a record, we have data about what we knew in the first instance, to recognize that we’ve learned something and what we’ve learned and that we might have been misinformed or unclear in the beginning. And I’ve suggested to my colleagues, one of the reasons that students don’t value their education as much as sometimes we think they should, or value our course as much as they think they should, is because when they get to the end, they may think “I always knew that, that was obvious, I knew those things.” And so we can help students realize that there’s a start point. And then there are many points along the road to learning, and that they may have learned more than they thought they did in a given course or given program. And I think that’s important for us in terms of demonstrating the value of education. And I think it’s important for students in terms of understanding and developing some humility about how it is that we learn and how hard it is and how long it takes to learn anything that’s really valuable.

Rebecca: I think, in my experience, students really appreciate the opportunity to slow down and reflect on their learning, because they’re often not given that opportunity. And when you’re doing that throughout an entire semester, they really do get excited about how much they’re learning and they get energized by that and seeing that they can achieve things over and over and over again.

Tom: I agree, and what we found with most of the studies of classroom assessment techniques, the simple ones is: first, classroom assessment techniques can’t really be separated from everything else that’s going on in teaching, they’re just one little part of the whole teaching and learning spectrum. They, at best, seem to make about a 5- 8% difference in how much students learn, which is about half a grade. And I think that’s important, but I don’t want to overstate it. So they’re not a panacea and they’re not a silver bullet, but they’re something that can lift up, to some degree, students’ performance and their learning. And the last thing I’ll say is nothing works for everybody. And so there is no technique or no series of techniques that every single teacher or every single student that I’ve worked with found valuable. And so one of the points of having 55 techniques is to hope that there’s one or two or five in there that people will actually find that work for them.

Rebecca: So you just reminded us about there’s 55 techniques, how do you imagine an instructor using your book? Is it a read from the beginning to the end? Do you have it organized in ways that instructors can easily dip in and out?

Tom: I can’t imagine reading it from the beginning to the end, and I don’t think anyone ever has. That would surprise me, except maybe the proofreaders [LAUGHTER] and they weren’t worrying about what it meant. So it’s organized so that there are many ways, there are actually eight ways to find a technique that might be useful for a given situation or a given teacher. One of those ways is the simplest and that’s alphabetical. So if someone has mentioned a technique to you, you can easily find it that way. The others are a bit more complicated, but I think the most valuable for many people is an index of all the techniques that have been used by discipline and documented by discipline, and the examples by discipline in the book. And there are a couple of 100 examples in the book of how people use these in different disciplines, kind of from A to Z, from anthropology to zoology, and everything in between. So those are ways that many people use. There are also a couple of inventories in the book. One is something called the teaching goals inventory, which was in the first and second edition, and has been used in lots of ways by many people and written about and research has been done on the teaching goals inventory, and it is exactly what it sounds like, an inventory of what we think we’re trying to teach. For this book, I created what I think is kind of a more contemporary take on that, and that’s something that I call the course learning outcomes inventory, or CLOI, and it’s really flipping that and saying, okay, it’s important what we want to teach, but let’s look at it from the point of view of what do we want students to learn? And so the CLOI, the course learning outcomes inventory, the statements are statements of learning outcomes. There are 57, they’re divided up into kinds of learning outcomes, and people can use it as a self assessment. Those items both in the teaching goals inventory and the CLOI are all linked to classroom assessment techniques in an index. So if you pick number 47 goal or number 47 outcome, you can find techniques that are linked to that. Lastly, we’ve done indexing by Bloom’s Taxonomy, and by Fink’s taxonomy of significant learning outcomes, and also by the big solo taxonomy, which is one that’s much used in the Commonwealth world, in the British and Canadian and Australian and New Zealand systems. So there are eight ways to skin a CAT in this book, or to find a cat at least. And I hope those will be useful to folks. And I think different kinds of indexing work for different kinds of people.

John: You mentioned that when this book first came out these things were done on paper. But one of the things that’s grown quite a bit over the years, as you’ve mentioned, is online instruction. Could you tell us a little bit about how some of the techniques can be used in an asynchronous environment?

Tom: Well, I think there are huge advantages to the technology that we have now compared to having to do all this on paper, having to read all those papers after class, or all those index cards, in my case, after class and then waited until the next class, when I saw students again, the time lag was a real problem. So being able to respond almost immediately, or in relatively short time to students in asynchronous courses, to their questions, and to their confusions, and to also let them know that they’re learning some things really well… they really got that… and that they’ve come up with some ideas that I never would have thought of that are excellent. To do those things in real time, I think, is much more effective in terms of teaching and learning than waiting two or three days or a week. Also, as you mentioned, we have technologies for polling students and for analyzing their responses that are ever more sophisticated and powerful. And so those save teachers a lot of time and work. I was always going through things counting and deciding on the number of times people had mentioned things on paper. This can all be done and is all done for you if you want to do it. And that, again, allows teachers to focus on what really matters. And that is: what’s the meaning of this data? What are students telling me? …and to make decisions about when they’re going to respond and what they’re going to respond to, because you can’t respond to everything in data if you have more than two students. And most of us did, and do. So I think that asynchronous learning has huge advantages, it has some downsides, but it has huge advantages. And one of those probably is that it makes it more difficult to assume that by looking at students, you can tell that they’re learning or not, which I’ve always found an odd idea, but many of my colleagues at least used to seem to think that if students were nodding and looking at them, that they were just tracking with them and understood everything that they were saying. And I used to say to my colleagues, “I’ve known many smilers and nodders that probably were thinking of something else entirely, and classroom assessment proved that to me.

John: I’ve seen that so much. We’ve heard so many faculty during workshops over the years who say, “Well, I can tell whether they’re learning just by looking at the expressions on their faces.” And I’ve never found that. There are a lot of students who will be nodding and smiling. And then when you ask them a question, or they ask you a question, it’s really clear that they don’t have the basic idea. At an economics conference, I was presenting a paper and there was someone sitting in the first row taking furious notes while I was talking. So I went up to the person after and asked them what they were taking notes about, because I was curious. And they were actually preparing for a session later in the day. But they looked really engaged throughout the whole thing.

Tom: They were engaged, just not with your lecture. [LAUGHTER]

John: Yes.

Tom: So I think the converse is also true. And that is their students, and people, who don’t appear to be paying any attention at all, or who are frowning or who have had their head down, who are actually paying total attention and learning a lot. And that’s one of the things I learned as well, early in my career, was not to make assumptions based on those sort of superficial impressions. And it’s hard not to, but when you actually learn that some students who appear not to be engaged are engaged in their quiet way. And that’s a humbling lesson, or it was for me.

Rebecca: One of the things that I really like about your book, and also the way the collection of the CATS all in one place is the flexibility, the flexibility of navigate the book, the flexibility of options for the instructor, but also the flexibility of options for learners as well, in different contexts. So I just wanted to highlight that flexibility piece that we haven’t really talked about quite yet.

Tom: Well, that’s the hope. And if this is created as an ebook, and I expect it will be, then I think that will offer people even more flexibility and more sort of usability. It will be a more user friendly book. And again, there are huge advantages to the kinds of technologies that we have available now that we didn’t then. So I think the real key to this is one of the axioms in the book. And that is it’s really about adapting, not adopting. So there are many, many tools in there that people have adapted in a variety of ways, renamed, and that’s all fine. And that’s all exactly what should happen. You should find them, adapt them to their uses, and if they don’t like the names of the CATs, they can change the name.

John: Would this be a good subject for a faculty reading group where people from different disciplines could get together and talk about how either they’ve applied these things or how they might apply them in their classes?

Tom: Well, it has been from the beginning. And we began this whole process working with groups of teachers, initially in community colleges and state colleges, who were interested in assessing student learning. And, of course, the point being trying to improve student learning. And what we did basically was use the teaching goals inventory at that time, you could use the course learning outcomes inventory, with those teachers to get them to focus on what they really wanted to know. And in the book, I’ve written about, in one of the chapters, four different ways you can approach using classroom assessment techniques and figuring out which one to use. But everybody had ideas and theories and questions or problems that they wanted to try to solve. And so we started with that. And then we said, if that’s your question or problem or theory or outcome or goal, what might be a good tool to try to help you gather a reasonable amount, a manageable amount, of data from students that might help you learn, and might help them learn? And that’s really where we began, and those discussions and the collegiality that came from them, and the interactions that came from that, were always the most valuable part of that professional development. That when we ask people: “What did you take away from this?” The first thing that they said was not a bunch of techniques. The first thing that they said was the interactions with my colleagues, the relationships with my colleagues, learning from my colleagues, and then they talked about the actual content that they had learned and the things that they had done. So I think those interactions and those relationships that people can make, by having a group focused around this. This is an excuse to get together and talk about teaching and learning. I think from our experience, we found any excuse to get folks together to talk about teaching and learning, especially if it’s across disciplines, is always fruitful.

Tom: …particularly if there’s food. [LAUGHTER]

John: Yes. The third edition list you as author with Todd Zakrajsek, could you tell us how this collaboration came about and how it worked?

Tom: I think this was, in part, an insurance policy for the publishers. Given my advanced age, they wanted to make sure that they had someone involved in the project, who could take over if I just kicked the bucket. So, in the last six months of the book project, after I had finished writing the manuscript, Todd Zakrajsek joined with me and was really invaluable in finishing the process, in coming up with great ideas and suggesting great ideas, especially for the use of technology and AI with CATs, and in helping me manage the process with the publisher, which is much more complicated than it was 30 years ago. He also wrote the discussion questions for use by teachers and faculty developers, and I think those are going to be very helpful to folks.

John: We’re recording this in May, and the book is scheduled for release in June. We always end by asking, what’s next?

Tom: Well, I’ll answer that. But I’ll first say to teachers, what I always say, and that is, “Feel free to ask your librarian to order the book, so that you can use it, and your colleagues can use it.” Don’t tell Wiley and Jossey Bass that I said that, but I think that’s important. So what’s next for me is I’ve been working for a while with a group of people in, strangely enough, pharmacy education. I’m not a pharmacist, and I’m doing research with graduate students on what we call core concepts in developing what are the core concepts in pharmacotherapy? And that’s been very interesting for me. I’ve learned a lot from that. And so I’m ongoing with that. And we’re beginning to work on educational materials related to that. So that’s one of the things that I’m doing. And I’ve actually, I don’t want to commit to this too much, but I’m thinking about writing a book based on an article that I wrote that people have found useful, called “A Teacher’s Dozen.” And so I’ve updated that. I’ve been working on that actually for a few years, and something may come of that.

Rebecca: Well, we’ll be excited to find out what that is soon, maybe. But you haven’t committed, but maybe. [LAUGHTER]

John: Well, thank you. It’s been great talking to you. And my copy of the book is on preorder. And I am looking forward to receiving that.

Tom: So am I. [LAUGHTER]

Rebecca: Well, thank you so much for joining us. It was great to hear about the origin story of the book and how it’s evolved over time and I know a whole other generation of teachers will appreciate the new edition as well as teachers who have been practicing for a long time.

Tom: I hope it will be useful. Listen, it’s been a pleasure and a privilege to talk with you both. Thank you for giving me this opportunity

John: We should note that this is the first of two podcast episodes discussing this book. In next week’s podcast we’ll be joined by Todd Zakrajsek, who will be discussing his role in this work.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]