309. Preparing Students for an AI Future

New technology is often seen as a threat to learning when first introduced in an educational setting. In this episode, Michelle Miller joins us to examine the question of when to stick with tools and methods that are familiar and when to investigate the possibilities of the future.

Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University.  She is the author of Minds Online: Teaching Effectively with Technology and Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World. Michelle is also a frequent contributor of articles on teaching and learning in higher education to publications such as The Chronicle of Higher Education.

Show Notes

Transcript

John: New technology is often seen as a threat to learning when first introduced in an educational setting. In this episode, we examine the question of when to stick with tools and methods that are familiar and when to investigate the possibilities of the future.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology and Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World. Michelle is also a frequent contributor of articles on teaching and learning in higher education to publications such as The Chronicle of Higher Education. Welcome back, Michelle.

Michelle: Hey, it’s great to be here.

Rebecca: Today’s teas are: ….Michelle, are you drinking tea?

Michelle: I’m actually still sticking with water. So it’s a healthy start so far for the day.

Rebecca: Sounds like a good plan.

John: I have ginger peach black tea today.

Rebecca: And I’ve got some Awake tea. We’re all starting the day [LAUGHTER].

John: So we’ve invited you here to discuss your August 17th Chronicle article on adapting to ChatGPT. You began that article by talking about your experience teaching a research methods course for the first time. Could you share that story? Because I think it’s a nice entree into this.

Michelle: Oh, thank you. I’m glad you agree. You never know when you’re sharing these kinds of personal experiences. But I will say this was triggered by my initial dawning awareness of the recent advances in AI tools, which we’re all talking about now. So initially, like probably a lot of people, I thought, well okay, it’s the latest thing and I don’t know how kind of attentive or concerned I should be about this. And as somebody who does write a lot about technology and education, I have a pretty high bar set for saying, “Oh wow, we actually kind of need to drop everything and look at this,” I’ve heard a lot of like, “Oh, this will change everything.” I know we all have. But as I started to get familiar with it, I thought “Oh my goodness, this really is a change” and it brought back that experience, which was from my very first assignment teaching the Research Methods in Psychology course at a, well, I’ll just say it was a small liberal arts institution, not my graduate institution. So I’m at this new place with this new group of students, very high expectations, and the research methods course… I think all disciplines have a course kind of like this, where we kind of go from, “Oh, we’re consuming and discussing research or scholarship in this area” to “Okay, how are we going to produce this and getting those skills.” So it is challenging, and one of the big challenges was and still is, in different forms, the statistical analysis. So you can’t really design a study and carry it out in psychological sciences without a working knowledge of what numbers are we going to be collecting, what kind of data (and it usually is quantitative data), and what’s our plan? What are we going to do with it once we have it, and getting all that statistical output for the first time and interpreting it, that is a big deal for psychology majors, it always is. So students are coming, probably pretty anxious, to this new class with a teacher they haven’t met before. This is my first time out as the instructor of record. And I prepared and prepared and prepared as we do. And one of the things that I worked on was, at the time, our methodology for analyzing quantitative data. We would use a statistics package and you had to feed it command line style input, it was basically like writing small programs to then hand over to the package. And you would have to define the data, you’d have to say, “Okay, here’s what’s in every column and every field of this file,” and there was a lot to it. And I was excited. Here’s all this knowledge I’m going to share with you. I had to work for years to figure out all my tricks of the trade for how to make these programs actually run. And so I’ve got my stack of overheads. I come in, and I have one of those flashbulb memories. I walked into the lab where we were going to be running the analysis portion, and I look over the students’ shoulders, and many of them have opened up and are starting to mess around with and play around with the newest version of this statistics package. And instead of these [LAUGHTER] screens with some commands, what am I looking at? I’m looking at spreadsheets [LAUGHTER]. So the data is going into these predefined boxes. There’s this big, pretty colorful interface with drop down menus… All the commands that I had to memorize [LAUGHTER], you can point and click, and I’m just looking at this and going, “Oh no, what do I do?” And part of my idea for this article was kind of going back and taking apart what that was like and where those reactions were coming from. And as I kind of put in a very condensed form in the article, I think it really was one part just purely sort of anxiety and maybe a little bit of loss and saying, “But I was going to share with you how to do these skills…” partly that “Oh no, what do I do now?” I’m a new instructor. I have to draft all this stuff, and then partly, yeah, curiosity and saying, “Well, wait a minute, is this really going to do the same thing as how I was generating these commands and I know you’re still going to need that critical thinking and the top level knowledge of “Okay, which menu item do you want?” Is this going to be more trouble than it’s worth? Are students going to be running all the wrong analyses because it’s just so easy to do, and it’s going to go away.” So all of that complex mix is, of course, not identical to, but I think pretty similar to how I felt… maybe how a lot of folks are feeling… about what is the role of this going to be in my teaching and in my field, and in scholarship in general going forward?

Rebecca: So in your article, you talk a lot about experimenting with AI tools to get started in thinking about how AI is related to your discipline. And we certainly have had lots of conversations with faculty about just getting in there and trying it out just to see how tools like ChatGPT work to become more familiar with how they might be integrated into their workflow. Can you share a little bit about how you’d recommend for faculty or how you were thinking about [LAUGHTER] jumping in and experimenting and just gettin g started in this space?

Michelle: Well, I think perhaps, it also can start with a little bit of that reflection and I think probably your listenership has a lot of very reflective faculty and instructors here. And I think that’s the great first step of “Alright now, if I’m feeling worried, or I’m feeling a very negative reaction, where’s that coming from and why?” But then, of course, yeah when you get it and actually start using it the way that I had to get it and start using my statistics package in a brand new way, then you do start to see, “Okay, well, what’s great, what’s concerning and not great, and what am I going to do with this in the future? So experimenting with the AI tools, and doing so from a really specific perspective. When I started experimenting at first, I think I thrashed around and kind of wasted some time and energy initially, looking at some things that were not really education focused. So something that’s aimed at people who are, say, social media managers, and how this will affect their lives is very different than me as a faculty member. So make sure you kind of narrow it down, and you’re a little planful about what you look at, what resources you’re going to tap into, and so on. And so that’s a good starting point. Now, here’s what I also noticed about my initial learning curve with this. So I decided to go with ChatGPT, myself, as the tool I wanted to get the most in depth with. So I did that and I noticed really that, of course, like with any sort of transfer of learning situation, and so many of those things we do with our students, I was falling back in a kind of an old pattern. So my first impulse was really funny, it was just to ask it questions, because I think now that we’ve had several decades of Google under our belts and other kinds of search engines, we get into these AI tools, and we treat them like search engines, which for many reasons, they really are not. Now, this is not bad, you can certainly get some interesting answers. But I think it’s good to really have at the front of your mind to kind of transition from simply asking questions to what these tools really shine with, which is following directions. I think one of the best little heuristics I’ve seen out there, just very general advice, is: role, goal, and instructions. So instead of coming in and saying “what is” or “find” or something like that, what perspective is it coming from? Is it acting as an expert? Is it acting as an editor? Is it going to role play the position of a college student? Tell it what you’re trying to accomplish, and then give it some instructions for what you want it to do. That’s a big kind of step that you can get to pretty quickly once you are experimenting. And that’s, I think, real important to do. So we have that. And of course, we also want to keep in mind that one of the big distinguishing factors as well is that these tools have memory, your session is going to unfold in a particular and unique way, depending not just on the prompts you give it, but what you’ve already asked it before. So, once you’ve got those two things, you can start experimenting with it. And I do think coming at it from very specific perspectives is important as I mentioned because there’s so little super general advice, or discipline-independent advice that I think is really going to be useful to you. And so doing that, I think a lot of us we start in a sort of a low-stakes, tentative way with other interests we might have. So for example, one of the first things that I did to test it out myself was I had it work out a kind of a tedious little problem in knitting. So I had a knitting pattern, and there’s just a particular little counting algorithm where to put increases in your pattern that always trips us up. And I was about to like, “Oh, I gotta go look this up,” then I thought “You know what, I’m gonna see if ChatGPT can do this.” And it did that really well. And by doing that in an area where I kind of knew what to expect, I could also push its parameters a little bit, make sure is this plausible? is what it’s given me… [LAUGHTER] does that map onto reality? and I can fact check it a little bit better as I go along. So those are some things that I think that we can do, for those who really are starting from scratch or close to it right now.

John: You’re suggesting that faculty should think about how AI tools such as this… and there’s a growing number of them, it seems more are coming out almost every week…, how they might be useful in your disciplines and in the types of things you’re preparing students for, because as you suggested it’s very different in different contexts. It might be very different if you’re teaching students to write than if you’re teaching them psychology or economics or math. And so it’s always tempting to prepare students for the way we were prepared for the world that we were entering into in our disciplines. And as you suggest in the article that we really need to prepare students for the world that they’re going to be entering. Should people be thinking about how it’s likely that students will be using these tools in the future and then helping prepare them for that world?

Michelle: Yeah, that’s a really good way to start getting our arms around this. In kind of the thinking that I’ve been doing and kind of going through this over the last couple of months… that just absolutely keeps coming up as a recurring thing, that this is so big, complicated, and overwhelming, and means very different things for different people in different fields. Being able to kind of divide and break down that problem is so important. So, yeah, I do think that and, for example, one of the very basic things that I’ve made some baby steps towards using myself is, ChatGPT is really good at kind of reformulating content that you give it, expanding or condensing it in particular. The other day, for example, I was really kind of working to shape a writing piece, and I had sort of a longer overview and I needed to go back and kind of take it back down to basics and give myself some ideas as a writer. So I was not having it write any prose for me. But I said, “Okay, take what I wrote and turn it into bullet points” and it did a great job at that. I had a request recently from somebody who was looking at some workshop content I had and said, “Oh, we really want to add on some questions where people can test their own understanding.” And you know, as the big retrieval practice [LAUGHTER] advocate and fan of all time, I’m like, “Oh, well, that’s a great idea. Oh, my goodness, and I’m gonna have to write this, I’m on a deadline.” And here too, I got, not a perfectly configured set of questions. but I got a really good starting point. So I was able to really quickly dump in some text and some content and say,”Write this many multiple choice and true/false questions.” And it did that really, really well. So those are two very elementary examples and some things that we can get in the habit of doing as faculty and as people who work with information and knowledge in general.

Rebecca: I’ve used ChatGPT, quite often to get started on things too, and generate design prompts, all kinds of things and have it revise and add things and really get me to think through some things and then kind of I do my own thing. But I use that as a good starting point to not have a blank page.

Michelle: Absolutely. Yeah, the blank page issue. And I think where we will need to develop our own practice is to say, “Okay, make sure we don’t conflate or accidentally commingle our work with ChatGPT’s, as we figure out what those acceptable parameters are.” But that reminds me too, I mean, we all have the arenas where we shine and the arenas where we have difficulty as, again, as faculty, as working professionals. I know graphic design is your background. I’m terrible. I’m great at words, but it reminds me, one of the things that I kind of made myself go and experiment with was creating a graphic, just for my online course that’s running right now, which would, for me, that would typically be a kind of an ordeal of searching and trying to find something that was legitimate to use and a lot of clipart, and I had it generate something. Now, I do not advise putting in like “exciting psychology image in the style of Salvador Dali,” [LAUGHTER] and seeing what comes out. He was not the right choice. It was quite terrifying. But after a lot of trial and error, I found something that was serviceable and there too, it’s not like I need to develop those skills. If I did, I would go about that very, very differently. But it’s something that I need in the course of my work but it’s a little outside of my real realm of expertise. So helpful there too. So yeah, the blank page… I think you really hit on something there.

John: Now did you use DALL-E or Midjourney or one of the other AI design tools to generate that image?

Michelle: Oh my goodness. Well, here again, [LAUGHTER] I was really out of the proverbial comfort zone for myself is really going to show. I did use DALL-E and I really wrestled with it for a couple of reasons. And so, as a non-graphic person, it did not come easily to me. Midjourney as well, if you’re not a Discord user, you’re really kind of fighting to figure out that interface at the same time and those that are familiar with cognitive load concept of [LAUGHTER] “I’m trying to focus on this project, but all this other stuff is happening. And then I had a good friend who’s a computer engineer and designs stained glass as a hobbyist [LAUGHTER] and kind of took my hand and said, “Okay, here’s some things you can do.” It actually came up with something a lot prettier, I have to say.

John: You had just mentioned two ways in which faculty could use this to summarize their work or to generate some questions. Not all faculty rely on retrieval practice in an optimal manner. Might this be something that students can use to fill in the gaps when they’re not getting enough retrieval practice or when they’re assigned more complex readings then they’re able to handle.

Michelle: Yeah, having the expertise is part of it, and I think we’re going to see a lot of developing understanding of that really cool tradeoff and handoff between our expertise and what the machine can do. I’m kicking around this idea as well, so I’m glad you brought that up. A nice side effect could be a new era for retrieval practice, since that is something of a limiting factor is getting quality prompts and questions for yourself. It’s funny, one of the things that I did do while taking a little prompt engineering course right now to try to build some of these skills and the facility with it. And one of the things they assigned was a big dense article [LAUGHTER] on prompt engineering, which was really great, but a little out of my field, and so I’m kind of going “Well, did I get that?” And then I thought, I better take my own medicine here and say, “Well, what’s the best way to ensure that you do and to find out if you don’t have a good grasp of what you were assigned?” And I was able to give it the content, I gave it, again, a role, a goal, and some instructions and said “Act as a tutor or a college professor, take this article, and give me five questions to test my knowledge. And then I told it to evaluate my answers [LAUGHTER] and see whether it was correct.” So that was about as meta as you can get, I think, in this area right now. So I’ve done it. And here again, it does a pretty good job, actually an excellent job. Do you want to use it for something super high stakes, probably not, especially without taking that expert eye to it. But wow, here’s something, here’s content that was challenging to me personally. It did not come with built in retrieval practice, or a live tutor to help me out with it. I read it, and I’m kind of going, “I don’t know, I don’t have a really confident feeling.” So I was able to run through that. And so yeah, that could be one of the initial steps that we suggest to students as a potentially helpful and not terribly risky way of using these really powerful new tools.

Rebecca: One of the things that this conversation is reminding me of and some of the others that we’ve had about ChatGPT is we have to talk a little bit about how students might use it in an assignment or something, or how we might coach a student to use it. But we don’t often talk a lot about ways that students might just come to a tool like this, and how they’re just going to use it on their own without us having any [LAUGHTER] impact. I think, often we jump to conclusions that they’re gonna have a tool write a paper or whatever. What are some other ways that we can imagine or roleplay or experiment in the role of a student to see how a tool like this might impact our learning?

Michelle: So that is another kind of neat running theme that does come up, I think, with these AI tools is role playing. I mean, this is what it’s essentially doing. And so having us roleplay the position of a student or having it evaluate our materials from the perspective of a student, I think, could be useful. But, it kind of reminds me let’s not have a total illusion of control over this. I think, as faculty, we have a very individualistic approach to our work. And I think that’s fine. But yeah, there’s a lot happening outside of the classroom that we should always have in mind. So just like with me on that hyper planned first course that I was going to be teaching, it just happened and students were already out there experimenting with “Oh, here’s how I can complete this basic statistics assignment with the assistance of this tool I’m going to teach myself. So that could be going on, almost certainly is going on, out there in the world of students. And it’s another time to do something which I know I have to remind myself to do, which is ask students and really talk to them about it. Early on, I think there was a little bit of like, “Oh, this is a sort of a taboo or a secret and I can’t talk to my professors about it and I want to broach it and professors, they didn’t want to broach it with their students because we don’t want to give anybody ideas or suggest some things are okay where they’re not. But I think we’re at a good point to just kind of level with our students and ask them “How do you think we could bring this in?” I think next semester, I’m going to run maybe an extra credit assignment and say, “Oh, okay, we’re gonna have a contest, you get bragging rights, and maybe a few points to “What is a good creative use of this tool in a way that relates to this class? Or can you create something, kind of a creative product or some kind of a demonstration that in some way ties to the class?” And I’ve learned through experience when I’m stumped, and I don’t quite know where to go with a tool or a technique or a problem, take it to the students and see what they can do with it.

Rebecca: I can see this is a real opportunity to just ask the students, how are they using it, and then take a look at the results that it’s creating. And then this is where we can provide some information about how expertise in a field [LAUGHTER] could actually make that better why that result is in what they think it is.

Michelle: Absolutely, and some of the best suggestions that I’ve seen out there, I’m kind of eagerly consuming across a lot of disciplines as much as I can to look at those suggestions. The most intriguing ones I’ve seen are kind of with things with a media literacy and critical thinking flair that tells students “Okay, here’s something to elicit from your AI tool that you’re using, and then we, from our human and expert perspectives, are going to critique that and see how we could improve it. So here too, critical thinking and those kinds of evaluation skills and abilities are some of the most prized things we want students to be getting in higher education. And they are simultaneously. for many different reasons, they are some of the hardest. So if we can bring that to bear on the problem, I think that can be a big benefit.

John: In the article, you suggested that faculty should consider introducing some AI based activities in their classes. Could you talk a little bit about some that you might be considering or that you might recommend to people?

Michelle: One of the things that I am going to be teaching, actually for the first time in a very long time, is a writing in psychology course, which has the added challenge of being fully online asynchronous, so that’s going to be coming up pretty soon for me. It’s still under construction, as I’m sure a lot of our activities and a lot of things are that we’re thinking about in this very fluid and rapidly developing area. I think things like outlining, things like having ChatGPT suggest improvements, and finding ways for students to also kind of track their workflow with that. I do think that one of the things that in our different professional [LAUGHTER] lives, because as I mentioned in the article, I think that should really lead the way of what work are we doing as faculty and as scholars in our particular areas. One of the things we’re going to have to be looking at is alright, how do I manage any output that I got from this and knowing what belongs to it and what was generated by me. What have I already asked it? If they’re particularly good prompts, how do I save those so I can reuse them? …another really good thing about interacting with the tools. But, I’m kind of playing around with some different ideas about having students generate maybe structures or suggestions that they can work off of themselves. And having CHATGPT give them some feedback on what they’ve developed so far. So one of the things you can ask it to do is critique what you tell it, so [LAUGHTER] you can say, “Okay, improve on this.” And then you can repeat, you can keep iterating on that, and you can keep fine tuning in different areas. You can also have it improve on its own work. So once it makes a suggestion you can, I mean, it’s virtually infinite what you can tell it to go back and do: to refocus, expand, condense, add and delete, and so on. So that’s kind of what I am shaping right here. I think too, at the Introduction to Psychology level, which is the other level that I frequently teach within, I’m not incorporating it quite yet. But I think having students have the opportunity or option to create a dialogue, an example, maybe even a short play or skit that it can produce to illustrate some concepts from the book and there ChatGPT is going to be filling in kind of all the specifics, the student won’t be doing it, but it’ll be up to them to say, “Well, what really stood out to me in this big, vast [LAUGHTER] landscape of introductory material that I think would be so cool to communicate to another person in a creative way?” And this can help out with that. I’m also going to be teaching my teaching practicum for graduate students coming up as well. And, of course, I want to incorporate just kind of the latest state of the art information about it. But also, it’s supposedly, I haven’t tried it myself yet, but supposedly it’s pretty good at structuring lesson plans. We don’t do formal lesson plans the way they’re done in K through 12 education, of course, but to give it the basics of an idea and then have a plan that you’re going to take into a course since that’s one of the things they do in that course is produce plans for courses and I gotta say it’s not a critical skill, the formatting and exactly how that’s all going to be laid out on the page, is not what they’re in the class to do. It’s to really develop their own teaching philosophy, knowledge, and the ability to put those into practice in a classroom. So if it can be an aid to that, great, and I also want them to know what the capabilities are if they haven’t experimented with them yet, so they can be very aware of that going into their first classes that they teach.

Rebecca: When you mentioned the example of a writing intensive class that’s fully asynchronous online, I immediately thought of all of the concerns [LAUGHTER], and barriers that faculty are really struggling with in really highly writing intensive spaces, and then fully online environments, especially around things like academic integrity. Can you talk a little bit about [LAUGHTER] some of the things that you’re thinking about as you’re working through how you’re gonna handle AI in that context?

Michelle: As I’m been talking with other faculty right now, one of the things that I really settled on is the importance of keeping these kind of threads of the conversation separate and so I’m really glad we’re kind of piecing that out from everything [LAUGHTER] else. Because once again, it’s just too much to say, well, on the one hand, how to prepare students and give them skills they might need in the future? How do I use it to enhance learning and oh my gosh, is everybody just going to have AI complete their assignments? It’s kind of too much at once. But once we do piece that out, as you might pick up on that I’m a little enthusiastic about some of the potential, does not mean I don’t think this is a pretty important concern. So I think we’re gonna see a lot of claims about “Oh, we’re going to AI proof assignments and I think probably many of your listeners have already run across AI detection tools and the severe problems with those right now. So I think we have to just say right now, for practical purposes, no, you cannot really reliably detect AI written material. I think that if you’re teaching online especially, I think we should all just say flat out that AI can take your exams. If you have really conventional exams, as I did before [LAUGHTER] this semester in some of my online courses, if you’ve got those, it can take those. And just to kind of drive home to folks, this is not just simple pattern matching, looking up your particular question that floated out into a database, no, it’s processing what you’re putting in. And it’s probably going to do pretty well at that. So for me, I’m kind of thinking about, in my own mind, a lot of these more as speed bumps. I can put speed bumps in the road, and to know what speed bumps are going to at least discourage students from just dumping the class work into ChatGPT. To know what’s effective, it really helps to go in and know what it does well and what it really stumbles on, that will give you some hints about how to make it less attractive. And that’s kind of what I’m settling on right now myself, and what I’ve shared with students, as I’ve spoken with them really candidly to say I’m not trying to police or catch people, I am not under an illusion that I can just AI proof everything. I want to remove obvious temptation, I want to make it so a student who otherwise is inclined to do the right thing, wants to have integrity and wants to learn doesn’t go in feeling like, “Oh, I’m at a disadvantage If I don’t just do this, it’s sitting right there.” So creating those nudges away from it, I think, is important. And yeah, I took the step of taking out conventional exams from the online class I’m teaching right now. And I have been steadily de-emphasizing them more with every single iteration. I think those who are into online course design might agree well, maybe that was never really a good fit to begin with. That’s something that we developed for these face-to-face environments, and we just kind of transplanted it into that environment. But I sort of ripped off that [LAUGHTER] bandaid and said, “Okay, we’re just not going to do this. I’ve put more into the other substance of the course, I put in other kinds of interactions. Because if I ask them Psychology 101 basic test questions, even if I write them fresh every time, it can answer those handily, it really can.

John: Recently, someone ran the Test of Understanding in College Economics through with the micro and macro versions. And I remember on the macro version ChatGPT-4 scored at the 99th percentile on this multiple choice quiz, which basically is the type of things that people would be putting in their regular tests. So it’s going to be a challenge because many of the things we use to assess student’s learning can all be completed by ChatGPT. What types of activities are you thinking of using in that online class that will let you assess student learning without assessing ChatGPT’s or other AI tools’ ability to represent learning?

Michelle: Well, I’ll share one that’s pretty simple, but I was doing anyway for other reasons. So just to take one very simple example of something that we do in that class, I really got an a big kick with Kahoot!, especially during the heyday of fully hybrid teaching where we were charged, as faculty, I know at my institution, where you have to have a class that can run synchronously with in-person and remote students at the same time, and run [LAUGHTER] asynchronously for students who need to do their work at a different time phase. And that was a lot and Kahoot! was a really good solution to that. It’s got a very K through 12 flavor to it, but most students just really take a shine to it anyway. And it is familiar to many of them from high school or previous classes right now. So it’s a quiz game, runs a timed gamified quiz. So students are answering test questions in these Kahoot!s that I set up. And because it has that flexibility, they have the option to play the quiz game sort of asynchronously on their own time, or we have those different live sessions that they can drop in and play against each other and against me. So that’s all great. But here’s the thing, prior to ChatGPT, I said I don’t want to grade this on accuracy, which feels really weird, right, as a faculty member to say, well, here’s the test and your grade is not based on the points you earn for accuracy. It’s very timed, a little hiccup in the connectivity you have at home can alter your score, and I just didn’t like it. So what students do is for their grade, they do a reflection. So I give the link to the Kahoot!, you play it, and then what you turn into me is this really informal and hopefully very authentic reflection, say, “Well, how did you do? What surprised you the most? Were there particular questions that tripped you up?” And also kind of getting them to say, “Well, what are you going to do differently next time?” And for those who are big fans of teaching metacognition, I mean, that comes through loud and clear, I’m sure. So every single module they have this opportunity to come in and say, “Okay, here’s how I’m doing, and here’s what I’m finding challenging in the content.” Is it AI proof? Absolutely not. No, it really isn’t. But it is, at least I think at that tipping point where the contortions you’d have to go through to come up with something that is gonna pass the sniff test with me, and if I’ve now read 1000s of these, I know what they tend to look like. And Kahoot!s are timed. I mean, could you really quickly transfer them out and type them in? Yes. It’s simply a speed bump. But the time would make that also a real challenge to kind of toggle back and forth. So I feel good about having that in the class. And so it’s something again, I’ve been developing for a while, I didn’t just come up with it, fortunately, the minute that ChatGPT really impinged on this class, but it was already in place. And I kind of was able to elevate that and have that be part of it. And so they’re doing that. I do a lot of collaborative annotation, I continue to be really happy with… I use Perusall. I know, that’s not the only option there is, but it’s great. They’ve got an open source textbook. And they’re in there commenting and playing off each other in the comments. So that is the kind of engagement I think that we need in force anyway, it is less of a temptation. And so I feel like that’s probably better than having them try to quickly type out answers to, frankly, pretty generic definitions and so on that we have in that course. Some people are not going to be happy with that, but that’s really truly what I’m doing in that course instead.

John: Might this lead to a bit of a shift to more people using ungrading techniques with those types of reflections as a way of shifting the focus away from grading, which would encourage the use of ChatGPT or other tools to focus on learning, which might discourage it from being used inappropriately?

Michelle: What a fantastic connection. And you know what? When I recently led a discussion with faculty in my own department about this, that is actually something that came up over and over just, it’s not ungrading, because not everybody is even kind of conversant with that concept. But how there are these trends that have been going on for a while of saying, you know, is a timed multiple choice test really what I need everything to hinge on in this online course. Ungrading, this idea of kind of, I think there’s this emerging almost idea I’ll call both sides-ism, or collaboration between student and teacher, which I think was also taking root through the pandemic teaching and that came to the forefront with me of just saying, “Okay, we’re not going to just be able to keep running everything the same way traditionally it’s been run,” which sometimes does have that underlying philosophy of, “Okay, I’m going to make you do things and then you owe me this work, and I’m going to judge it and you’re going to try to get the highest points with the least effort. I mean, that whole dynamic, that is what I think powers this interest in ungrading, which is so exciting, and it’s gonna maybe be pushed ahead by this as well. Ultimately, the reason why you’re going to do these exercises I assign to you is because you want to develop these skills. You are here for a reason, and I am here to help you. So that is, I think, a real positive perspective we can bring to this and I would love to see those two things wedded together, especially now that tests can be taken by ChatGPT, then, we should relook at all of our evaluation and sort of the underlying philosophy that powers it.

John: One of the concerns about ChatGPT is it sometimes makes mistakes, it’s sometimes will make stuff up, and it’s also not very good with citations. In many cases, it will just completely fabricate citations, where it will get the authors of people who’ve done research in the field, but will grab other titles or make up other titles for their work. Might that be a way in which we could could give students an assignment to use one of these tools to generate a paper or a summary on some topic, but then have them go out and verify the arguments made and look for citations and document it just as a way of helping prepare them for a world where they have a tool which is really powerful, but is also sometimes going off in strange directions, so that they can develop their critical thinking skills more effectively.

Michelle: Yeah, looping back to that critical thinking idea. Could this also be a real way to elevate what we’ve been doing and give us some new options in this really challenging and high value area? And yes, this is another thing that I think faculty hopefully will discover and get a sense of as they experiment themselves. I think probably a lot of us have also experimented with, just ask it about yourself. Ask it, what has Dr. Michelle Miller written? There’s a whole collaborator [LAUGHTER] I have never heard of, and when it goes off the rails, it goes. And it’s one thing to say really kind of super vaguely to say like, “Oh, AI may produce output that can’t be trusted.” And that has that real like, okay, caution, but not really, feel to it. That’s a whole other thing to actually sit with it and say, alright, have it generate these citations. They sure do look scholarly, don’t they really look right? Okay, now go check them out. And say, this came out of pure thin air, didn’t it? Or it was close, but it was way off in some particular way. So as in so many areas, to actually have the opportunity to say, okay, generate it and then look at it, and it’s staring you right there in the face some of the issues. So I think that we will see a lot of faculty coming up with really dynamic exercises that are finely tuned to their particular area. But yeah, when we talk about writing all kinds of scholarly writing and research in general, I think that’s going to be a very rich field for ideas. So I’m looking forward to seeing what students and faculty come up with there.

Rebecca: That’s a nice lead into the way that we always wrap up, Michelle, which is to ask: “what’s next?”

Michelle: Well, gosh, alright. So I’m continuing to write about and disseminate all kinds of exciting research findings. I’ve got my research base substack, that’s still going pretty strong. After summer. I actually focused it on ChatGPT and AI for a couple of months. But now I’m back to more general topics in psychology, neuroscience, education, and technology. So articles that pull in at least three out of four on those. I’ve got some other bigger writing projects that are still in the cooker. And so I’ll leave it at that with those and I’m continuing to really develop what I know about and what I can do with ChatGPT. As I was monitoring this literature, it was really very clear that we are at a very, very early stage of scholarship and applied information that people can actually use. Those are all things that are very much on the horizon for my next couple of months.

Rebecca: Well, thank you so much, Michelle, we always enjoy talking with you. And it’s always good to think through and process this new world with others.

Michelle: Absolutely.

John: It certainly keeps things more interesting and exciting than just doing the same thing in the same way all the time. Well, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

307. Career Readiness

Students do not always understand how the work that they do in our classes helps prepare them for their future careers. In this episode, Chilton Reynolds and Ed Beck join us to discuss one institution’s approach to helping students understand and articulate how their course learning activities intersect with career competencies. Chilton is the Director of the Faculty Center for Teaching, Learning, and Scholarship at SUNY Oneonta. Ed is an Open and Online Learning Specialist, also at SUNY Oneonta. Chilton and Ed have both worked on integrating career readiness skills into the curriculum.

Transcript

John: Students do not always understand how the work that they do in our classes helps prepare them for their future careers. In this episode, we discuss one institution’s approach to helping students understand and articulate how their course learning activities intersect with career competencies.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Chilton Reynolds and Ed Beck. Chilton is the Director of the Faculty Center for Teaching, Learning, and Scholarship at SUNY Oneonta. Ed is an Open and Online Learning Specialist, also at SUNY Oneonta. Chilton and Ed have both worked on integrating career readiness skills into the curriculum. Welcome Chilton and Ed.

Chilton: Hey, It’s nice to be here.

Ed: Thanks, John.

Rebecca: Today’s teas are… Chilton, are you drinking any tea today?

Chilton: I am. It’s afternoon here, so I’ve moved to iced tea. I make my own decaffeinated, slightly sweetened, peach iced tea for the afternoon.

Rebecca: Sounds nice and refreshing.

Chilton: Yes.

Rebecca: What about you, Ed?

Ed: I am drinking a Chamomile honey and vanilla tea, in a very fancy special mug.

Rebecca: Oh, that’s a Tea for Teaching mug. I wonder where you got it.

John: And I am drinking an Irish Breakfast tea today.

Rebecca: Also in a tea for teaching mug. I have Lady Grey, I think.

John: We’ve invited you here today to discuss your work at SUNY Oneonta in making explicit connections between course learning objectives and career readiness skills. Can you tell us a little bit about that?

Chilton: Yes, we’d love to. And again, thanks for having us. We’re excited to be here to share this project that we’ve been doing. We just finished up with our first cohort where we are really trying to help our students make really clear connections between what’s going on in the classroom with career competency skills that they’ll be using after they leave college. So the focus of this program was really on helping faculty build into their courses, times to allow students to reflect on what they’re doing in the classroom, and really say explicitly: “Here’s the skill that we are trying to build towards. Here’s what we’re doing in class. Now, as a student, practice actually making that connection. We want you to either write that and think about it in writing or say it out loud, and practice saying out loud, so that those connections can become as strong as possible after you leave this class.” The focus of this was on lower-level classes, we specifically targeted lower-level classes, because we thought that by the time they’re getting to their senior seminars, they’re doing that in the class already. But we don’t have these conversations in our 1000- and 2000-level classes. And so the more we can do this in our lower-level classes, hopefully when they get to those upper-level classes, they can say, “Oh, yes, I do remember talking about technology skills or communication skills early on and I can make connections now between what happened in that class and what’s going on.”

Ed: Yeah, the big thing that we always are talking about in the instructional design field, in the faculty development field, we’re talking about authentic learning all the time. But I joke sometimes when I say like if a student completes a course built on authentic learning, but can’t talk about it in an interview, or articulate it to themselves or others, did it really happen? And this is our practice. This is us saying, “if we’re going to do all the effort to make sure that our courses are built on authentic learning, we’re building authentic tasks into them. Let’s go ahead and do the next step of reflection, of practicing, so that students are prepared to speak about it.”

Rebecca: Can you talk a little bit more about how you rolled it out to faculty, because you’re talking about working on it through the center, and then getting faculty to adopt it in these lower level classes? Can you talk a little bit about those details?

Chilton: Yes. So we had started with a call for faculty. We actually had gotten a grant, there was local money from our institution to be able to do this where our incoming president had created… we didn’t have a strategic plan at the time, so he created a initiative called “regaining momentum” that was very much focused on re-engaging our students, both incoming students and current students. And so one of the focuses was on career readiness. And so that was kind of “how do we help our students make those connections?” So we had applied for a grant, we received the grant, and in doing that had promised that we would do this over three years, the first year being our first cohort. So we actually put out a call for proposals and went to a couple of faculty that we knew were doing some of this from some of our previous work, and said, “Would you be willing to be a part of this?” and then also have the full call for everybody across campus, and we were looking for 10 faculty, and I think we had 11 proposals to begin with. After that we went through the team that was built from across campus. We can talk about that in just a second too. But they had a team that would review those proposals and then said, “Yes, we had a cohort of 10,” which is what we had funding for the first year of the cohort, and then went through the process with them over the year.

Ed: In our center, we’ve really been thinking about how we can focus on the student experience. We’re in a transition phase right now, we used to be known as the Teaching, Learning, and Technology Center, and we are transitioning, and as of July 1st, we’re now the Faculty Center on campus. We were thinking about how we stop leading with technology. We always were thinking about teaching but we wanted to lead with that. And one of the things that we were doing was we were focusing in on the AAC&U high-impact practices. And we went through that long list of high-impact practices and said “okay, what fits into the work that we are already doing as a center?” and kind of identified some of them, so we had already been doing sessions and cohorts of project-based learning with our faculty members, we had already been investigating and helping build ePortfolios. And we always saw ourselves as the collaborative learning people. So what we wanted to do was create a cohort of people that were thinking about this and tie it to a goal that we could keep coming back to, and have these faculty meet with each other throughout a semester to really create a community around a central idea. And that’s where the idea really came from was to keep reconnecting through the semester and focus on building that community versus the sometimes one-off presentations that faculty development can sometimes feel like.

Rebecca: Can you talk a little bit about the career readiness competencies that you’re focusing on?

Chilton: Yeah, when we started this application process, we were connecting with other groups across campus. And one of those that when we talk about career development should be the Career Development Center. So we reached out to them and they talked about how they were using the NACE career readiness competencies. NACE is a national organization that is connecting what’s going on in the classroom to careers afterwards. There are eight competencies that are a part of those and they align a lot of their work specifically with that. Additionally, we found out that some of our co-curricular activities also aligned with the NACE competencies. We have a Lead program, which is a leadership program on our campus, and that uses the NACE competencies as well. The School of Liberal Arts has a program going on right now where they were trying to do a lot of this similar work outside of the classroom with students helping them connect what they had been doing in their classes with what was going on through the NACE competencies. So we found there was a lot of work already happening on campus, and so we really wanted to make sure that we aligned with that as well. What we like about NACE competencies is it really aligns with a lot of the work that goes on in our classrooms. And that’s what resonated with us, and that said, we’re focused on what’s going on in the classroom, how we can help support faculty in doing more useful work inside the classroom. And so we thought about how the NACE competencies really do that. So we think about things like professionalism, communication, critical thinking, teamwork, technology, and leadership. Then there’s equity, inclusion, and career and self development. Those are the soft skills is that word that got used a lot in the past to kind of say, “Yes, we do these things, but we didn’t really help students make those connections between what’s going on.” So we felt like it was a great framework to take into the classroom and say, you’re doing this as faculty, you know, you were doing this, but the students don’t always know that they’re doing this, how can we be able to help do that? The other thing I’ll follow up with that is, as we were exploring this more, we reached out to the POD Network, and actually found out [LAUGHTER] from SUNY, there was already work going on some of this as well. So our Center for Professional Development, has a whole certificate program that’s around connecting career readiness skills into the classroom and our use of the NACE competencies as a part of that as well. So it was really a lot of tie-ins that we saw really strong connections between what was happening on our campus and things that were happening locally.

John: We have talked about that, to some extent in our previous podcast with Jessica Krueger, and we’ll include a link to that, in the show notes.

Chilton: And one thing I follow up with that, John, is we have a couple of pre-professional programs. And this seems to fit really well there, like career readiness makes sense when you have a pre-professional program that’s preparing you for a specific program. We were also trying to reach into our liberal arts programs, into our science programs, into lots of other programs that might not be as focused on a specific profession, but still are connecting into these career readiness competencies.

John: And since we’re doing these things in the classes anyway, it’s nice for students to be able to recognize that these are skills that are going to be helpful for them in their future careers. And when they can see that, I think that may help provide a little more intrinsic motivation to engage in these practices and develop those skills. How have students responded to that?

Chilton: So we are in the first year of this, and this is one of the things we were reflecting on as we were preparing for this in that we realized our first year was focused on what faculty are going to be doing. As Ed said, we’ve been working with some faculty on this, that have been doing this on a smaller scale. But as far as this program, we’re looking forward in year two to really hearing from students and hearing how that’s going to go, so we’ll have to provide some feedback and liner notes later on to let you know when we hear about from the students.

Ed: Yeah, I’m gonna lead a committee to do the IRB and create some surveys to send out to students that are part of the program and have a little bit more of that student voice that we can report back on. Because I think that’s really important. It grew out of a proposal like that, that I’ll talk about a little bit later. But we had done that student interviews and student feedback once before, that really helped create this framework that we were really trying to set up with now a cohort of faculty members.

Rebecca: I really love hearing that you’re using NACE across your institution in different spaces. So you mentioned that Career Development Center is doing it as well as your center. Can you talk a little bit about how that collaboration is working?

Chilton: Yeah, so we really see this as a partnership. And it’s one of the things that we really tried to be intentional early on. Because when you say career readiness, that is a Career Development Center thing, and we don’t want there to be any perception that we’re trying to take over what they’re doing and we want to be able to just support them so that when students come to them, they are more prepared. A part of the original proposal was going to the Career Development Center and saying, we want to do this with you, would you be willing to partner with us? We can do more of this in the classroom. It was very much a partnership, it was very much us wanting to say, “What is it that you do in the Career Development Center, and then also, where can we help support you?” And then make sure that we feed into what you’re already doing. So it’s not any appearance of us coming in and trying to take over your programming, but just help our students be more prepared when they come to your program.

Ed: Actually, we had a great day at a winter workshop where the Career Development Center sat with our faculty and said, “Here are some of the things that we are already doing, here are the services that we’re purchasing, here are the things that we do at one-on-one consultations, here was what it could look like if you invited us into your course.” And some of our faculty members did that and invited the Career Development Center into their course to speak to them. And some of our faculty members were doing other things that incorporated the competencies but didn’t necessarily incorporate an outside group like the Career Development Center. So we had a wide range, even among our cohort of what they were doing. During that winter workshop that I was referencing, we actually brought in an outside trainer. And that was really nice. Chilton mentioned that the SUNY Center for Professional Development, the CPD had already been doing a four course sequence on the NACE competencies, which was really meant for a variety of professionals, it wasn’t just faculty, but the instructor that came highly recommended to us was Jessie Stack Lombardo also from SUNY for the SUNY Geneseo Career Development Center Director. And she came in and did some workshops with us and the faculty thinking about what are the small things that we can do in our class that helped students reflect, that helps students make those connections?

John: Could you tell us a little bit about the impetus for starting this program?

Ed: Yeah, so even before the program, we were, of course, working with wonderful faculty members here at SUNY Oneonta. And one of the things that we’d been doing quite a bit was thinking about making websites and ePortfolios, having opportunities for students to build their own web space, build their own web presence. So even before the cohort happened, we had one great instructor that said, “Hey, I would really love to be thinking about building ePortfolio projects into my course, would you help me do that?” During this time, John, you know, we were doing the SUNYCreate, a domain of one’s own initiative. We were giving websites to students. That was a technology-focused initiative. But we were doing a lot of these things already. I said, “Yeah, let’s come in. Let’s do that.” I was invited into that class several times. And we were so proud of this course, the way it came out. I want to give such a big shout out to Dr. Sarah Portway. She later went out and won the Chancellor’s Award for Excellence in Teaching based on a lot of the work that she was doing. They were building a fashion magazine online, the students were taking the articles they submitted for that fashion magazine, and also bringing them back to their portfolio and showcasing them on their own sites in addition, and she said, “Hey, why don’t we take this thing on the road? Why don’t we go to the AAC&U’s Institute on OER and ePortfolios.” And we said, “Okay, let’s do an IRB, get some student feedback from that, to bring to the conference so that we have that student voice when we go through.” And the feedback was fantastic. Students really responded to it. It was a wonderful presentation. But we were also starting to realize during those interviews, it wasn’t a negative, but it wasn’t all positive. Students were still not making all the connections between the skills that they had done and things they had practiced and the skills they had acquired, and being able to articulate that. I have this memory of a student saying to me, “I wish I could have put this on my portfolio, but it was a group assignment, so I can’t put a group assignment on my personal portfolio.” And I remember just kind of stopping the interview format and saying to her, “Oh, I would absolutely put that group assignment on your portfolio if you’re proud of it. I would absolutely describe what it is you did to contribute to that. group atmosphere and talk about how you can be a successful collaborator and describe how you work in team environments. And then put that thing that you’re proud of, that artifact that you’re proud of, on your portfolio, but also with the framing of what it means for you to be a good teammate and what it means for you to be a good collaborator.” And the student said “Oh, I never thought about it like that, I guess I could do something like that.” And Dr. Portway, being a fantastic instructor, never being satisfied with how things went in the last class was kind of like, “We need to think about this a little bit more. We need to be more explicit. We’re already doing all these authentic assignments. And at some level, it’s hitting. And we definitely want to keep going down this road. But on some level, we are missing something in helping those students make those connections. What do we need to do in the classroom activities, in the way that the assignments are presented, that helps walk them through that to make them just a little bit more prepared, because the authentic skills were already in the course. They just needed help making that connection.” And that was really the thing for me that I walked back from that experience and knocked on Chilton’s door and said, “We need to be doing more of this, and we need to be doing it not one at a time, but with groups of faculty members.” And that was really important to me.

Chilton: And what was interesting to me was that to carry that on a little bit more, when we first had this proposal ePortfolios was in the proposal title. We were really focused on we want to do ePortfolios for everyone. And some of the feedback we received was “Yes, ePortfolios can be a part of that but this could be a much wider conversation,” which is again, how we got back to NACE. There are these bigger frameworks that we can be a part of. So we went from, “Yes, here’s this great tool to no, no, no, let’s look at it from a framework perspective.” And now we’re at the point where we’re like, yes, some of the projects will be ePortfolios, some of them will be other things. And that’s okay. And that was bigger than the tool. This is about helping our students think about what they’re doing and helping them connect to things that will be useful for them after they leave college.

John: One quick follow up, you mentioned that you have groups of faculty who worked on it. As I understand this, this was a faculty learning community that you put together, where faculty received some slight funding or a small stipend as part of the participation. Have you done any work there with entire departments in revising their curriculum yet?

Ed: No, we haven’t done the departmental level work yet. Right now we’re focusing on coalitions of the willing, having faculty who are interested in these types of things. One of the hopes is that after doing three cohorts, having worked with multiple faculty one year, then the second year, then a third year, new faculty each year that we can get to a point where we’re ready to have a bigger discussion. There was one participant in the group that was really focused on making a freshman ePortfolio with the explicit reason to keep contributing to it throughout the program. And I think that shows a lot of promise. But we’ve still got to do some work to get the buy-in from the rest of the department to make sure that it gets used. So I mean, there is a lot to be done there. And it’s one of the things I’m hopeful for the future.

Chilton: To add on to that, one of our goals out of this is to be able to build a repository where we can share and our hope is that we can have enough examples that when we go to a department, we can say, here’s some small changes you can make. Ed had mentioned this earlier, we want to be able to have a breadth of here’s some small tweaks you can make. Or here’s some larger things you can do. And be able to have some examples that are multidisciplinary, that are a wide range of both implementation needs, as well as examples from different departments so that when they go to a department, we can say this doesn’t have to be a large change, it could just be helping make some small changes to help those students make connections.

Rebecca: So I wanted to follow up on an earlier point that you were making, from an experience that I’ve had as an instructor, and I’m sure many other instructors have had is you work really hard to make these kinds of career-ready activities, things like professional email writing, and portfolio projects, and team projects, the list goes on. We do many of these things as instructors, and then you inevitably have this conversation, a one-on-one maybe with a student. And you just realize they have no idea why they were doing any of the things and you’re like, “Oh, I failed the student clearly [LAUGHTER]. I could have done a better job.” And so it seems like frameworks like NACE could be really helpful, both for instructors and students to just be more explicit about those things and to practice talking about them. Can you talk a little bit about that piece of the puzzle?

Ed: Yeah, I think it’s so important to have small opportunities to embed a skill or embed a practice in there. So I’m going to start off with a very small thing that I think anybody could throw into their class. At the end of the course, it’s at reflection time, we want to talk about what you learned. Let’s take a moment and think about a common interview technique is the star interview technique, you’ve probably heard of it, where you describe a situation, then you have the task that you were assigned, the action that you took, and then a result, explain that, say “Hey this is how a lot of times we make sure that we have an action-oriented response to an interview question.” Now talk about this course, using the STAR method. What situations did your instructor put you in? What were you asked to do? What did you do in order to be successful in there? And then is there anything else you want to share about it? Are there next steps that you should continue to do that your instructors put you on the path to? Or are there things that you’ve realized about yourself that you need to continue on with for the future for the next thing? That’s so simple, it’s not rewriting an entire course. Yet, it’s a little opportunity to say, this is important, and what we did had meaning, and take a moment to integrate that into your context. How will you talk about this course in the future.

Chilton: What was interesting to me when we were doing this was when we first started out, we list a whole bunch of sample outcomes that get at what you were talking about, I’m going to do this email, I’m going to have them do this thing and it’s going to be great. And as we got to the conversation with our faculty, we realized that what we were missing was really creating the places for the students to practice making the connection. We have to practice the skills all the time, we’re like, “Yes, we do this.” As the faculty member, we understand that there is a connection between this and career readiness but unless the students are actually practicing making the connection, not just doing the action, but making the connection, then it doesn’t always stick for them. And so that’s where we started to shift from, what do we want the faculty to do? How do we want the students to practice this so that it does stick for them? So it is meaningful for them in a way that they can think about it again, hopefully, a year, two years from now, when they’re finishing their college career and starting to think more about career readiness. That was a shift for us of what is the faculty member going to do to how do we help the students really intentionally practice what they are doing, practice talking about what they have done, and making that connection to, in this case, the NACE framework, because we thought it was such a good framework to talk about.

Ed: I feel like we’re saying NACE too often. So I feel like it’s always helpful to be a little more specific. So let’s talk about communication. We’re teaching communication to students all the time. One of the key aspects is audience. So have the conversation with your students. When we communicate to different audiences, we use different standards. So part of the reason why I’m asking you to write a more formal paper, in research style format, is I want you to be prepared to speak to other experts in your field. But when we shift to the oral presentation, I want you to adjust your language, so that you’re speaking to a non-expert, you’re speaking to your future colleagues, you’re speaking to a potential customer. And when you make that switch, make sure it’s intentional. And then at the end of this course, I’m going to ask you to reflect on that to think about what choices you made when you were speaking to someone who you expect to already understand and be embedded into the discipline and someone who you do not expect. How is it different when you talk to a colleague versus when you talk to your friends and family about what you do. That’s an important communication competency. So let’s talk about it and intentional choices that we can be making.

John: How many faculty members and how many departments were involved in this project so far?

Chilton: So we had our first cohort, and in that cohort, we specifically targeted to having 10 faculty. But we were very specific about trying to have faculty from as many departments and as many schools as possible. So we have three schools on our campus, we ensured that we had representation from all three schools, we ensured that we have representation from multiple departments. So in the end, we had nine different departments as a part of this. We did have overlap in one department, from two of our participants. As we said before, we did focus and said in the call that we wanted you to work with a 1000- or 2000-level class. And so that was part of the call as well. So we actually had a couple people that applied for this that were planning on doing this in a 3000-level class. We reached out to all of them and said “Do you have any lower-level classes that could be part of this?” Two of them said yes and one didn’t, which was one person that we weren’t able to take in. We’re focusing on in years two and three, again, lower-level courses, and going to try to continue to have faculty from as many different departments as possible, so that when we get to the end of this, again, we have a nice repository of examples from as many different disciplines and as many different schools as possible.

Ed: And we can invite some of the cohort I faculty back as mentors, and we can incorporate them into year two in a different way, as we continue to try to build a larger community and push a conversation that we think needs to happen on campus.

Rebecca: Can you talk a little bit about what was expected of a faculty member who was accepted into the program?

Chilton: So we’ve spelled that out upfront, we had already been planning on our campus, what we call the SUNY digital learning conference that was focused on open and public education. And we purposely built in a track in there that was about career readiness. Originally, as we’ve been talking about, we were focused on ePortfolios, and so I thought a lot of them will be doing ePortfolios. But in the expansion of that, we wanted to make sure that we really talked about how we can make connections. So we said that we would pay for the faculty to be a part of that conference. So they attended that conference in November of 2022. And that was the first part, kind of the kickoff for this cohort. We then had a January full day workshop, as Ed had talked about earlier, brought in Jessie Stack Lombardo from SUNY Geneseo to be our speaker for that, and she wasn’t even a speaker she really planned the day and it was very highly interactive with those ten faculty. So as Ed said, we have a staff member from the Career Development Center was a part of that and presented locally, Jesse then talked about some different frameworks to be able to do including NACE and how you can start to think about both small changes that we made and large changes. And then we had said in the call that the expectation would be that by the end of the Spring 23 semester, they would turn in a revised syllabus and examples of work that they are doing to the group. We realized that wasn’t specific enough. So we then created a rubric that focused in on three specific areas of what they would need to do. First part of that rubric is what would be the changes they would make in their syllabus to really spell out what are the NACE competencies? Are you focusing on all of them? Are you focusing on one of them? But didn’t have to be a lot in there, but we did want to have it be addressed in their syllabus in some way. And then what is the activity they’re going to be doing where they actually have students practicing and how will the students receive feedback on that? Well, there’s three levels, it’s not present as a part of the rubric. And then we had two levels of “yes, it meets expectations.” And we were thinking again, what are those small changes that could happen, but then also, we had a what’s above expectation, where what’s something if you were really dreaming about what it could be, where could you take it and what could it be, so we kind of want to have “Yes, you meet expectations” that would help us get small changes that would be usable by everybody. And then what could this be look like if you really wanted to really [LAUGHTER] dive into the deep end with it and explore what could happen with it a little bit more, and make it so that it was better for students, not just in the class, but beyond the classroom.

Ed: Yeah, the only thing I’ll add to that Chilton is we also met once a month during the spring semester. So we had recurring meetings throughout the Spring semester. With those rubrics between the present and highly effective, and we’ll share the rubric so that you can put it in the notes if you’d like. We were starting to think about not only did you incorporate the NACE competency in your course, but you were also presented your prompts or your things in a way that gave the students the opportunity to think about future activities they could take, future things they would want to do. And that was really important to us as we were doing it is to not only just create a moment of reflection for the students at that moment, but also to make that connection of, okay now that I’ve had that moment of reflection, now what? Should I be picking out some different courses? Should I be finding an internship? Should I be doing something now to set myself up for success? And so we don’t get that panicked feeling when the student is at senior year and they go into the career readiness and or the Career Development Center and say, “Okay, what do I do now?”

John: What type of incentives were offered to faculty to participate in the program?

Chilton: So we spell that out as pay for their participation in the conference in November. So they were able to go for free and participate. It was on our campus that made it easy for them to be able to go. It was just their conference registration that we covered as a part of this. In addition, we paid them a stipend for attending the January workshop. So officially it was $90 was the stipend to attend the workshop. And then when they completed and turned in their final version of their revised syllabus and examples of activities, there was another $510 stipend. So in total, it was a $600 stipend. But as a part of that final revision, we actually did review their submissions, looked at the rubric and did give them feedback… a couple of people, we said, “Hey you’re missing…” and asked them to go back and do some additional work. So we did hold them accountable to that rubric before getting the final stipend. And so it was a useful and interesting conversation when the leadership team did meet to kind of look at those to be able to say, “What do we like about this? What are we thinking for cohorts two, and three? What might be asked for more specifically next time to make this even more meaningful for our students?” So we’re already starting to think about cohort two and looking forward to that for next year.

Rebecca: Can you talk a little bit about how faculty responded to their participation,

Ed: We take the faculty’s response and the feedback they gave us really seriously. We gave them the opportunity. They had the reflections that they were doing that, of course, we knew who was speaking. But we also gave them some opportunities to give us some anonymous feedback, so that they could tell us how they really felt about us. And we were just really pleased with it in year one. We do recognize that we have to keep honing our message, we have to keep defining what we mean by career readiness, and what we mean by incorporating it into class. We need to have our elevator pitch a little bit more refined and down. Because what’s evolved through this conversation is, we’ve really talked about the skills are already there, but we can be more intentional about it. And we can be intentional in the ways we ask students to reflect and practice in ways that we really believe can be beneficial for students. But that can still be a difficult conversation. When people see career readiness in 1000 and 2000 level classes, some people are bristled or turned off by that because they’re thinking, “Oh, just one more thing that I have to do.” Now we didn’t get that from our participants in the cohort that much, because they applied and they came here on purpose, that was nice to have a group that was really wanting to be here and was willing to try some things with us in this space that we were creating. But overall, I would say that the feedback had been very positive.

Chilton: Looking through the feedback from faculty, I just pulled out there was one quote that stuck out to me that i’ll read quickly that came from one of our professors in our communication arts department, where this professor said, “Students said that they felt more confident.” So this is actually one of the professors we recruited into this program that had been doing this already. This professor did have some experience with students doing something but said that “Students felt more confident in the skills as a professional, and were able to articulate how the experiences they had in my class connected to the expectations and employers would have of them. They also appreciated being told of why we had to do certain projects and to help them transition from college to life after college.” And so I think that really speaks to how the professors enjoyed having time to be able to do that.

John: We only have courses from 100 to 500 levels. It seems there has been a bit of a course number inflation there at [LAUGHTER] Oneonta. That was just a joke. I’m sorry.

Chilton: We were told that everybody in SUNY was moving to four course digit numbers and so we over the past two years, it was like this really big project that we did to move from three digit to four digit numbers, because at least we heard, everybody in SUNY is doing this. So we have to do it. It’s very intriguing that not everybody had to do that. [LAUGHTER].

Ed: And simultaneously, as we were going from three digits to four digits, we didn’t have 400 level classes previously. And the feedback we were getting was that that was seen as a deficiency by some people who are reviewing our students’ transcripts, even though calling all of our upper division 300 level, and that people applying to professional schools would get that explanation they would understand why there weren’t 400 level. Other people who are maybe not as skilled at reading a transcript are like, “Well, did this student avoid all 400 level work?” And so simultaneously as we were adding another digit, we were also transitioning to having 1000, 2000, 3000, and 4000 level classes. So that was a big change that took a lot of curriculum writing and mapping through over the last two years. It wasn’t just as easy as adding a zero on to every course.

Rebecca: Sounds like such a fun project. Sign me up. [LAUGHTER] So we always wrap up by asking what’s next?

Chilton: So we are excited about Cohort Two. We are going to be starting our recruitment in the fall. We actually have a fall faculty Institute on campus. This year is very much focused on what the communities of practice that are already happening on campus and how you can get involved. And so that’s going to be our first, not our first, but that’s gonna be one of our big recruitment pitches for Cohort Two. In Cohort Two, we are looking to be able to include more faculty from a wider range, we are going to be starting to get into faculty that might not have as much experience in doing this. So we are thinking about how we hone our pitch and how we focus this to a wider audience to be able to say “No, this is not big changes in your classes. This is just asking one additional question or allowing space for one additional time for students to be able to practice connecting what you’re already doing to these career readiness competencies.”

Ed: And I would say what’s next for me is this experience has really solidified the idea for me that we need to continue in faculty development centers, to make spaces where faculty can repeatedly come back and interact on the same topic, getting away from that kind of one and done workshop, and identifying major things that we want to return to through the year inviting people into that space to share. Because when those faculty, when they get an opportunity to think and show off what they’re doing, it really is a wonderful spread of ideas. And you get a lot from all the energy in the room.

Rebecca: Well, thank you so much for joining us and sharing how this project’s unfolded at your institution.

John: And you’re both doing some really great work at SUNY Oneonta and it’s great to keep in touch and thank you for joining us.

Chilton: And thank you. It’s a pleasure to be here with you. So thanks for taking the time to be with us today.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

300. Episode 300 Reflection

This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

Show Notes

Transcript

John: This is episode 300 of the Tea for Teaching podcast. Whether you are a new listener or have been with us for all 300 episodes, we are very grateful that you’ve joined us on our podcasting journey. In this episode, we celebrate this milestone by reflecting on what we’ve learned and how the podcast has evolved.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Now that we’ve reached Episode 300, we thought we’d take the opportunity to reflect on some of the changes that we’ve seen in higher education since we launched the podcast in 2017.

John: Today’s teas are:…

Rebecca: I have Blue Sapphire.

John: I heard you just stocked up on a trip to your favorite tea store.

Rebecca: Yes, I was really excited to stock up on my favorite and I have a couple new ones too. So maybe in some upcoming episodes, we can try those out.

John: And I have Bing Cherry Black tea from Harry and David’s which is made by the Republic of Tea. When we first got started on the podcast, much of the focus was on specific teaching practices and techniques and interesting projects. Most of our guests were people that we knew or guests who were within our professional networks.

Rebecca: in the spring of 2020, as we know, [LAUGHTER] the focus shifted to the challenges associated with remote and online teaching, and the challenges facing remote learners and instructors.

John: As we became accustomed to pandemic teaching, we focused a bit more on faculty concerns as we transitioned into the transformed higher ed landscape. Historically, higher ed had been designed to serve the elites of society, and while higher ed gradually became more open and students have become much more diverse, many residual practices have worked against serving the students that we have. During the pandemic, faculty became much more aware of the inequities facing our students as well as faculty and staff.

Rebecca: Yeah, so one of the things that we’ve been talking about quite a bit is this more holistic focus on the needs of our students and faculty as humans, and really generating and creating a much more inclusive higher ed environment. How do you see that moving forward, John?

John: One of the things we’ve talked about is addressing the needs caused by the increased demands on time for faculty, staff, and students. As we developed new teaching techniques and tried to build more structure into our courses, it put much more demands on faculty in terms of redesigning their courses, in terms of paying more attention to the needs of students, and providing students with more feedback. And that has led to issues with burnout, which we’ve addressed in a number of podcasts.

Rebecca: And you’ve never experienced that, have you, John?

John: The day is not over yet. [LAUGHTER]

Rebecca: Well, how do we think about supporting faculty as we move forward with all these demands on their time in trying to prevent burnout? We’ve talked about this in a couple of episodes, but as we enjoy summer and have a little bit of more downtime for some of us… maybe not you, John… and move into the fall, what are some things that we need to be thinking about for our own classes to prevent burnout?

John: One of the things that we’re trying to be careful with when we recommend new teaching techniques to faculty at the teaching center here, is that they change approaches gradually, that small changes, incremental changes, are much easier to accommodate than the type of rapid changes that people had to do when they first moved into remote teaching. And so I think we have to be careful in making sure that we maintain a balance and we don’t burn out ourselves, because we’re not going to be very effective in supporting our students if we’re struggling to get through each day ourselves.

Rebecca: Yeah, we need to be present, just like we want our students to be able to be present and have the supports around them to be present in their learning. I think one thing that we’re also talking about in grad studies in our office is really this increased stress on faculty, and how do we support faculty, but also how do we support graduate student populations through things like accountability groups, or ways where there’s another human for accountability, but also for support, and not necessarily a mentor model, where there’s a power dynamic, but really a peer-to-peer approach to connect people together.

John: And we’re running two reading groups this fall to address some of these needs. One of the reading groups is on Sarah Rose Cavanagh’s new book, Mind over Monsters. And the other one is the second edition of Jim Lang’s Small Teaching. We had done that a few years ago, but we’ve had a lot of new faculty since then. And while we try to reach as many faculty as we can in our workshops, there’s a lot of faculty who are still teaching in pretty much the same way as faculty were teaching a century or so ago. And we’re hoping that by encouraging small modifications in teaching approaches, it might encourage more faculty to participate in introducing active learning activities and evidence-based teaching approaches.

Rebecca: It’s really easy to slip back into past practices when we’re tired [LAUGHTER] and overworked. And it’s not surprising that people have kind of slipped back into assignments and stuff that they’re really familiar with to reduce the cognitive load around new stuff and the many stressors around. So having that added support to help faculty re-engage with some of those ideas is, I think, a really great idea at this juncture. And I love that Mind over Monsters is one of the reading groups as well, because mental health is such an increasing concern, not just for students, but also for faculty and staff.

John: And we’re very much looking forward to both of these reading groups. Among the things we’ve talked about more frequently since the start of the pandemic are the challenges faced by underrepresented and contingent faculty.

Rebecca: I think when we’re introducing new techniques, and we’re thinking about supporting students around mental health, or we’re thinking about evidence-based practices and engaging in active learning, we need to remember that contingent faculty or underrepresented faculty have different barriers or different obstacles in implementing these things… or even more pushback from students and implementing some of these techniques. So we really need to be cognizant of supporting each other and realizing that we don’t all have the same kind of supports in place. I think some populations of faculty are just overly criticized. And when they try something new, it’s not accepted in the same way that a more dominant group’s adoption of those same techniques might be.

John: And that’s true both by students as well as by their faculty peers. And one of the things that’s come up in many of the podcasts we’ve discussed are the biases in both student and faculty evaluation of teaching.

Rebecca: Yeah. One of the things that I think is on the minds of our faculty too, is, as we’ve seen increased diversity of our students, we’ve seen diversity in levels of preparation. And I think those inequities have always been there. But again, maybe it’s more visible now than it had been in the past. How do we work through that in our classrooms, especially in these more introductory classes as students transition into college?

John: Well, I think those inequities have always been there, but they certainly grew with remote teaching, because our students face very unequal resources in their school districts and in their households. And when people are physically in the classroom, they’re at least exposed to the same infrastructure within their institutions. But when students were taking classes from home, as we talked about in many, many episodes, during a pandemic, they had very unequal network access, they had very unequal computing facilities, they may have been sharing a computer with multiple family members, they may have been forced to work. And as a result, the inequities in prior education and prior learning became much more dramatic during the period of remote teaching. And that disproportionately affected students from low-income households and low-income school districts. And what we have to do is provide resources, I think, for all students to be successful. And while we always should have been teaching, or providing resources and support, for all students, those needs have become much greater now, because while we are bringing in a much more diverse student body, we’re also losing students who come in with less preparation at some of the highest rates we’ve ever seen before. And we have to make sure that we’re providing the students that we accept with the support they need to be successful. And there’s lots of ways of doing it, you can build in some additional resources, you can connect to YouTube videos, and such things and provide support to students, you can use mastery learning quiz systems, and many other techniques. But we have to work towards having more faculty building that in because while many faculty are doing these types of things, and trying to build more support and more structure into their classes, it’s not a universal phenomenon.

Rebecca: And maybe even acknowledging that some students in the class are quite literally working harder to get to the same level.

John: The last few years when I’ve been teaching my large intro class, that’s something I’ve mentioned explicitly. I said, everyone here has all the resources they need to be successful. But if you had taken an AP introductory microeconomics course, or something close to that, you’re not going to have to work as hard to attain mastery of many of the concepts. If you have not been exposed to these things, or if your background in working with math and using graphs is not as strong, you’re going to have to work a bit harder. And that’s not a message that a lot of students appreciate hearing. But if we want to get all of our students to the same level at the end, the students who do come in with a weaker background need additional support to get there. And using tutoring when available, encouraging students to come in and talk to their professors and use office hours, all of those things can help but we’ve got a ways to go. What are some of the things you do to try to provide support for the increasingly diverse student body?

Rebecca: Yeah, I think the reality is that what we give each student isn’t the same, because what they come in with is not the same. So I often am trying to assess where students are and then pushing them right at that moment where they’re at, rather than expecting everyone to be at the same point. And I can do that a little more efficiently in a small class than you can in a bigger class. But I think we need to use those smaller classroom spaces to be able to do that so that everyone feels challenged, but also has what they need.

John: My introductory course is a prerequisite for all upper-level economics classes, and most of my students will be moving on to upper-level classes, so they have to reach at least a minimum level of proficiency in the discipline in order to be successful in their future classes. In other classes, instructors can be more flexible, and just try to get the most learning gains in their students, no matter what their starting points were. In my introductory class, at least, I have to pretty much take the students where they are, and try to get them all to the same place, while making sure that they’re all challenged. And that’s a very challenging goal to reach.

Rebecca: …and the difference between teaching those introductory classes versus higher level classes within a discipline, for sure, I think one of the most efficient things we can do is making sure that all students know the most effective ways to learn, because they don’t necessarily know those things coming into college, or even into graduate school. What we need to just remember is learning isn’t something that we just magically know how to do, we need to learn strategies and techniques that are effective.

John: We’ve had a number of guests over the last year or two who’ve talked about books that they’ve provided, or resources they’ve created to help students be more successful. Because one of the things that’s been pretty obvious for quite a while is that the study techniques and the learning strategies that students use are not generally consistent with what evidence suggests is most effective. And as a result, students are not using their time as efficiently as they can, by engaging in strategies that they perceive as being useful, that really result in very little increase in long-term recall… strategies such as highlighting, repeated rereading, and so forth. And one of the things that might be helpful is if we all could shift students a little bit in the direction of using evidence-based learning strategies, and some of that could take place through course design, by building infrastructures that incentivize the use of these techniques.

Rebecca: Yeah, and I think the moment that students realize that they don’t have the most efficient way, or the moment that a student begins to struggle is different, depending on some of that background, that they have. Students that come in well prepared may have never really struggled in high school, and maybe eventually, maybe even in the first year of college, you don’t struggle, but maybe it hits a little later on in their education, maybe not until graduate school. And then other students might struggle the second they get to college, because there’s not as much structure in place as there was in high school. So I think we need to be underscoring these techniques at all levels, and not just in their first year.

John: And one other thing that’s been discussed in many podcast episodes, is the importance of making the hidden curriculum of higher ed transparent to students, so that we don’t expect students to know what a syllabus is or how it could be used, that we shouldn’t expect students to know what is expected on a term paper in a class without making those expectations explicit and transparent to students. Because in general, we see a lot of students coming in, and they see it as a game where they’re trying to guess at what instructors are asking. And many of those guesses, especially for students who have not been in college prep classes before, are wrong. And they wasted a lot of time and effort that could have been spent more productively developing their understanding of the subject matter.

Rebecca: And the reality is that there’s differences between disciplines and between courses. And so the more we can be explicit about expectations within our own discipline, and within our own courses, and beyond the classroom experience of higher ed, because there’s expectations in other spaces as well, like student clubs, athletics, and all of the rest of the co-curricular activities that support student learning are incredibly important. And those are also not obvious.

John: One of the things that we’ve talked about much more on the podcast, and higher ed in general has been addressing much more extensively since the pandemic, is alternative grading approaches. Because traditional grading approaches and traditional course structures generally incentivize students to cram and to focus on maximizing their grades, rather than maximizing learning, so that if we really want students to shift to evidence based learning strategies, it would be really helpful if we could shift students emphasis away from grades and faculty emphasis away from high-stakes assessed activity and shift it more to activities that result in deeper learning, more long-term learning. And we’ve talked to many guests who have shifted to using strategy such as specifications grading, mastery learning systems, portfolio assessments, and ungraving, which has become one of the most talked about topics in higher ed in the last few years.

Rebecca: Yeah. And I think one of the things that comes up in a lot of those conversations is concerns over students just wanting the right answer and not learning and not critically thinking about the subject matter and the knowing of why and how, and doing analysis. And I think every instructor [LAUGHTER] has a desire for some of those kinds of conversations to come out in their classes, rather than just regurgitation of things that they’ve said in class.

John: Part of the issue is that when we get students in college, they’ve already had 13 years of experience in K through 12, where grades were the primary area of focus. And as a result, it’s hard to shift that focus from grades to learning. Besides alternative grading, we might use some other strategies such as encouraging students to be more reflective on their work, to spend some time in reflection-based activities and metacognitive development type activities.

Rebecca: Yeah, I know, this is a space where I was maybe a little hesitant at first thinking like, “Oh, these are just quick assignments that have no meaning,” but quickly realizing actually the value in really good well designed reflective activities that challenge students to think through how and why they learned something and what it is that they actually got out of an activity. And I’m often very surprised about how much learning occurs that is not visible, despite the fact that I teach studio classes, so I’m with my students much more than the average instructor. So I actually do observe a lot of learning. But in the reflection activities, I’m hearing a lot about how students are spending their time or things that they really struggled with and worked through that I wasn’t aware of. It also helps me understand where they’re not aware [LAUGHTER] of their own learning, or where they’re using strategies that aren’t as effective and helps with interventions. I know you’ve done a lot around metacognition, especially in your lower-level classes, but also in your upper classes.

John: In at least a couple of my courses. I’ve been using the metacognitive cafe discussion forum, which was actually the topic of our second podcast, Judie Littlejohn and I jointly developed this quite a few years back. And it’s been remarkably effective. It’s basically a low-stakes discussion forum that I’m using in my online classes, where students will reflect on their learning and share their learning strategies and will also read a bit about retrieval practice and spaced practice and the benefits of sleep in learning. And every time I do it, even though it’s only a trivial portion of the grade, it’s 5% of their total grade for participating in that activity, the students report that it was the most valuable learning experience they had in the class. A large proportion of the students at the end of the terms say they wish that they had learned these things back in elementary school, that they had been using practices that were not efficient and they didn’t realize that because they’ve never been taught how to learn. And it’s something that students have found really valuable. And the other nice thing about it is because, in this particular case, it’s done in a discussion forum, it helps them build community and helps them get to know each other, because they’ll often talk about the challenges they face. In online classes, many of the students have families where they’re taking care of young children, they may be working different shifts, they may be faced with other challenges that normally wouldn’t come up in a content discussion forum in an online class. But when they share that, and they share those challenges, and they share their career expectations, and they talk about how what they’re learning might be useful in their expected careers, besides the sense of connection, it also helps students see the relevance of what they’re doing and sharing that with other students helps build a little bit more intrinsic motivation in learning.

Rebecca: It also seems like there’s a bit of an immediacy in that context as well, because the information can immediately be put into action in a real lived experience and not something that may feel abstract, which sometimes happens within a discipline when it feels like maybe it’s not a thing I’m going to do anytime soon, professionally. So I think this really highlights the reason why we need to help students hook into everything that we’re doing to make it feel like they have a personal, professional, or educational connection to their own goals.

John: One of the topics that I use in each class where I’ve done this, at a point where students face the first really challenging material in the class, is just asking them to discuss how they deal with challenges. They share useful strategies, but one of the main benefits of that is it normalizes the sense of struggle, that when students are struggling with concepts, they often feel that they’re alone on this, but when they hear that other people are struggling with exactly the same issues and exactly the same concepts, it normalizes it, and again, it helps them understand that challenge is an important part of learning, which is not the message that they’ve generally received throughout their prior educational experience before coming to college.

Rebecca: It seems to me like this is the same reason why our reading groups work so well for faculty development as well is this connection among peers, but also that the challenges we experience are not in isolation. [LAUGHTER]

John: One of the things I’ve always enjoyed about the reading group is when people will come up with a technique or describe something they’ve done in class and people in completely different disciplines who might never have considered that will make connections and see how they could do something similar in their classes. That understanding that we’re all facing the same challenges makes it much easier to deal with some of the day-to-day stresses that we might have in our classrooms.

Rebecca: I know that one of the things that has come up in our reading groups, and also in our conversations about the future of higher ed and where we’re going to be going over the next few years is student engagement, and then specifically, the role of AI tools like ChatGPT [LAUGHTER] in the conversation. So if students don’t feel motivated, and they’re relying heavily on these tools, how do we get students to re-engage with the idea of learning?

John: Well, going back even just a little before the introduction of ChatGPT, which kind of hit higher ed by storm in late November of 2022, we did see a dramatic increase in the use of sites such as chegg.com, and various other sites out there, where the use of those tools became normalized a little bit, which made it much more challenging to give online assessments. And I think that’s where most people are concerned right now about things like ChatGPT, because with other places, you could at least locate where answers were coming from. And you could address that with the students and attribute it to the specific sites where they got their answers, which was, again, a bit of a challenge. But ChatGPT is raising some challenges for assessment that are going to be difficult to deal with, because it’s much more difficult to determine who is the author of specific items submitted online for assessment. And a lot of people are struggling with that right now. I know I’ve been struggling with it. In my spring 2023 online class, the quality of student writing on essays improved fairly dramatically over the course of the semester. And that seemed to correlate with the spread of the use of ChatGPT a bit. AI tools are really powerful, and they can be really useful. And they have a lot of potential value in education and in providing support for personal and work productivity. Right now, I think, more people are focused on the challenges, but we’re going to have to start thinking about ways in which we can productively integrate this and prepare students for a world in which the availability of AI tools will be ubiquitous.

Rebecca: And you teach in some really challenging contexts, really large classes in person, a number of online sections, and I know ChatGPT is keeping you up at night. What are some things that you’re thinking about… maybe haven’t resolved… but that you’re really thinking about redesigning or rethinking or retooling in the fall to just respond to the moment that we are currently living in.

John: As of 2020, I had shifted all the quizzing to online quizzes and tests and midterm exams and so forth. I’m seriously thinking about in my large face-to-face class, moving back to at least in in-class midterm and an in-class final exam. I really appreciated the fact that I could let students do it at their own pace, and that it took some of the anxiety and stress away when students did not have this two-hour time limit to complete an exam in the classroom. But with the size of the class, a large proportion of the testing is done with multiple choice exams, or algorithmic questions, and those are types of things that ChatGPT answers really, really well. Not too long ago, someone posted that ChatGPT 4 received a score of a 99th percentile on the Test of Understanding in College Economics (the microeconomics version of that), and those are the same types of questions that I’d be giving students on these quizzes. And while I had 1000s of questions that I had created that students were selecting from, all of those questions now are vulnerable to the use of AI tools, which makes it much more difficult to assess in that large class. Right now, the only thing I’ve really thought about doing differently in my large class is moving back to at least a couple of in-class exams. Now some of the things I was doing, such as polling questions embedded in the class activities and working on problems in class, where students submit that in real time, are generally much less subject to that type of issue. I know there are tools where students can scan the questions and so forth, they get responses back a bit more quickly, but it wouldn’t be as easy for them to do in real time when they’re in a polling environment. One of the main benefits of that is when I use polling, it was always tied with peer-to-peer discussions. And those peer-to-peer discussions is where most of the learning actually occurred from those in-class problem-solving exercises. For my online class, I’m not sure what I’m going to do. One thing I have done in the past is I’ve had students do podcast projects. And again, it’s pretty easy for chat GPT to generate scripts, but these projects are pretty heavily scaffolded. Students submit a proposal and they go through a number of steps to get there. And projects that are scaffolded like that, are probably a little bit less sensitive to the use of AI tools to generate the entire project. What are you thinking about in terms of your classes, or in terms of the graduate program?

Rebecca: I think we’ve talked a lot about the concern over the validity of our assessments and wanting there to be accuracy, not just for our sakes, but for students’ stakes in the value of their degrees moving forward. Part of it, I think, is really engaging in conversations around ethics around these tools, and not necessarily discouraging the use of the tools entirely, or banning the tools. I think that just motivates people to want to use them more, but rather to use them in ways that are productive, or interesting, but are also well documented… [LAUGHTER] like students are disclosing what they’re doing. And we can analyze the use of the tools in particular ways because maybe it could save time in particular places and not take away from certain kinds of learning, as long as we keep the learning objectives [LAUGHTER] up front. And then we assess when we’re using particular tools to determine whether or not it’s taking away from the learning. But I think these are hard conversations to have, and certainly not things that I want to be policing.

John: And I’d much rather not be policing these things. Sometimes students haven’t given me much choice in that. One example that I’ve seen recently is students submitting exam responses that asked him to analyze recent data, where the response said something to the effect: “as an AI tool, I do not have access to this data.” And when a student submits work like that, it’s pretty clear that they haven’t even read the essay responses they’re submitting on that graded assessment activity. And we want to make sure that students do actually interact and engage with their learning materials. Perhaps we can also design assessments that are not as vulnerable to AI-generated text. This semester, with my online classes, one thing I have shifted to, instead of having them discuss general debates or issues in economics, I have them focus on interpreting videos online, for example, where economists are debating certain topics, or doing readings that are not in the training database for ChatGPT, which means it’s much harder for AI tools to generate responses when they don’t have access to the underlying content that’s the focus of the assessment activity.

Rebecca: Would hyperlocal situations or examples also be a strategy because there’d be less widely available information on something like that.

John: Definitely. Information on the local community or the campus community or other local things, information that would just not be part of the training database is a good place in which we can ask students to connect the materials their learning to real-world events so that you maintain that sense of relevance while ensuring that the students are actively engaging with the work themselves rather than using a tool.

Rebecca: One of the things that I’ve used historically in my design classes, and it’s a little easier again, because I teach studio classes and see students more often so I tend to have a hunch as to what they’re working on, because I’m seeing them working on things, is really documenting process and not just using language, but showing through a video and showing steps along the way that might not be as easy to capture as an end product using an AI tool.

John: In general, open pedagogy projects, too, could be less vulnerable to having work being done entirely by AI tools. So videos would be an example, wikis perhaps might be.

Rebecca: I think that things that combine text and image are more challenging to have an AI tool create, at this moment.

John: That may very well change…

Rebecca: …by the time this episode comes out. [LAUGHTER] I think one of the things that I’m hearing us say actually, is that a lot of the strategies to reduce intellectual integrity or academic integrity issues around ChatGPT are also the things that are more likely to engage students and foster their learning anyways because they’re more authentic assessments, they’re probably more project based, they’re probably more long term with milestones along the way. And these are things that students often deeply engage in. And I think when they can connect to their local community, whether that’s the campus or the community that campus is situated in, or even their own hometown, in different ways around the discipline, those are all ways that students get a hunger to want to learn more.

John: And going back to our earlier discussion of the importance of shifting students’ focus from grades to learning, students are using tools like ChatGPT to raise their grades, even though they recognize it does not support their learning. If we can shift students’ focus to recognize the value of learning as improving skills that they’re going to need later in life, that should reduce the incentive for students to use shortcuts to avoid learning material.

Rebecca: If we’re not just looking for the right answer, but the journey to an answer, and even if it’s an incorrect answer, being able to understand why it’s not correct, and allowing that to be the learning is a really different way than our education has historically worked. The future of higher ed seems really stressful, John. [LAUGHTER]

John: It does, but it always has. That’s nothing new. But certainly the last few years have seen a lot of rapid change that… I hate to use the word unprecedented… but that have been relatively unprecedented.

Rebecca: And I think it really does speak to this need to connect with other colleagues, where we can share some of the challenges that we’re facing and brainstorm together to improve our teaching, but also to improve the level of stress we’re experiencing. [LAUGHTER]

John: One of the other things that we’ve talked about, especially within the last year or so is growing faculty concerns over student engagement. When students first came back to the classroom, there was a lot of excitement about being back. But since then, faculty generally seem to be noting that the level of engagement of students has shifted or has changed somewhat. More students are not completing assignments. Students in some classes have been disappearing from class as the semester progresses. And there’s a lot of concern that students are not as fully engaged with their coursework as they had been prior to the pandemic. So Rebecca, how are we going to solve this?

Rebecca: That’s a good question, John. I think one of the things that this aligns with is the higher incidences of loneliness, and mental health. And so finding ways to connect students to each other, and establishing those peer networks, I think, is one of the most important things that we can do in our classes. And it’s something that I’ve maybe always done in some way. But I’m being much more intentional about moving forward, because I’m feeling like even if students are in the same room, they’re still feeling really isolated. And so we have to be intentional about creating those opportunities for students to experience connection and feeling like they want to show up for each other and for themselves,

John: …using more group activities in class where the work of each student depends on the contribution of the other members does help create that sort of pressure on students to be there for their peers, to be there for the rest of their team. And that could be very useful.

Rebecca: Yeah. And I think the key to that, though, is not just assuming that students know how to interact with one another, or how to depend on one another in a team context, but really scaffolding those learning opportunities that really start with making connections and establishing relationships, because it’s the relationship that’s going to cause the pressure to show up for someone. And I think when we’re seeing high incidence of like ghosting, for example, it’s because the individuals don’t feel connected to the people that they’re ghosting.

John: And there have been a lot of studies done recently that show the importance of a sense of belonging in student persistence. So helping students form those connections is really important, because we have so many students who go to college, build up a huge volume of debt, and then disappear without getting the degree which does serve as a signal that they’ve actually accomplished something as a result of their education. And they end up with more financial struggles than they would have had had they not started. So we do want to help students form those connections for their own sakes, for their own future success. And one of the books we used in a past reading group was Relationship-Rich Education by Leo Lambert and Peter Felton. And that summarizes a lot of the research on the importance of building community and building connections, and also provides some really nice examples of ways in which institutions can transform to help facilitate those connections.

Rebecca: As instructors, we have a lot of power in that space to help students feel a sense of belonging. We can do really simple things to make someone feel seen and if they feel seen, they’re more likely to feel like they belong. So personalized messages, getting to know your students a little bit, being approachable, calling students by name, all of those things help students feel like they’re a part of a particular community. There’s so much to still learn and to come together around. And so I know that we’re looking forward to having many more guests and many more conversations to help work through many of the questions and concerns and things that we’ve raised today and have been raised by our colleagues. Now, John, we always wrap up by asking what’s next?

John: Well, what’s next for me is I’m heading down to North Carolina to teach at Duke again next week. And I’m looking forward to this. And this time, I’m going to try avoiding getting run over by a car. So I can actually teach my classes down there and spend some time away from the hospital this summer,

Rebecca: #life_goals. [LAUGHTER]

John: Small goals are sometimes more achievable. And Rebecca, what’s next for you?

Rebecca: This summer, I’m looking forward to doing some more work on our graduate student online orientation, which we put together as we transition to our new course management system in the fall and also working with some colleagues on an accessibility online module.

John: And we’re looking forward to talking to more of our wonderful guests. I’ve really enjoyed the experience of interacting with so many great people doing some really good research and doing such good work in higher ed.

Rebecca: We’re grateful for all of our guests and all of our listeners. So thanks for listening

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

293. Study Like a Champ

The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, Regan Gurung and John Dunlosky join us to discuss a new resource they have created that is designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

Show Notes

  • Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and practice. Routledge.
  • Gurung, Regan and J. Dunlosky (2023). Study Like a Champ: The Psychology-Based Guide to “Grade A” Study Habits. American Psychological Association.
  • Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological science in the public interest, 9(3), 105-119.

Transcript

John: The study strategies that most students use may be helpful in passing high-stakes assessments, but do not generally support long-term recall of fundamental concepts. In this episode, we discuss a new resource designed to help students develop more efficient study strategies and improve their metacognitive and self-regulatory skills.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Regan Gurung and John Dunlosky. Regan is a social psychologist and is an associate vice provost and Executive Director of the Center for Teaching and Learning at Oregon State University, He is the author of over 120 peer-reviewed articles and has co-authored or co-edited 15 books. John is a professor in the Department of Psychological Sciences and the Director of the Science of Learning and Education Center at Kent State University. He co-authored the first textbook on metacognition and has edited several volumes on education. Regan and John are the co-authors of Study Like a Champ, which was published earlier this year by the American Psychological Association. Welcome, John and welcome back, Regan!

Regan: Thank you.

John D.: It’s great to be here.

Rebecca: Today’s teas are:… Regan, are you drinking tea?

Regan: Just water because it is a scorcher here in Oregon. So water, and lots of it.

Rebecca: Hydration is important. How about you, John?

John D.: No tea for me for about five years, I used to really be addicted to oolong. And now I’ve gone full espresso. [LAUGHTER] Water for me today too. It’s a little bit late for espresso on the East Coast.

John: And I have an Irish Breakfast tea today.

Rebecca: Oh, John, you’re really pumping up the caffeine today. Also on the East Coast,

John: My day started at 5:30 this morning…

Rebecca: So did mine.

John: …and it’s going to be going really late. We’re in the midst of grading here, we’re recording this a little bit earlier than it will be released. So we’ve invited you here today to discuss Study like a Champ. What prompted you to create this book?

Regan: So both of us have been teaching for a long time, both of us do research on teaching and learning. And I think both of us really like taking stuff from the lab and testing it in the classroom. And, I think, as passionate teachers, we noticed that we knew the stuff, we knew what students should be doing, we told our students what they should be doing. We even tried to design our classes in that way. But clearly, we needed to do more of it. And the impetus behind this book was let’s put these expert tips in the hands of students everywhere. We didn’t want it to be just those students who had teachers like us who talked about studying, we want it to be in student hands. And that’s where it came from, specifically driven by the fact that even though there’s some really neat stuff out there on learning, very little of that, if any, is written to the student. And that’s something right from the get go when John and I first talked, we agreed that this would be in a voice that would speak directly to students.

John D.: Absolutely. In fact, I think that was the main impetus because of all the wonderful books and volumes out there on learning sciences. And just too many of them, I think, appropriately so written for teachers. And often they tend to be a little bit fact listing because of that. In other words, I’m going to tell teachers everything there is to know about learning, which can be overwhelming. And what Regan and I wanted to do is kind of find the most effective and best little snapshots of the learning sciences to share with students, the things that we think will move the needle the quickest, so to speak. So by no means do we tell students everything there is to know about the learning sciences, but we hope we tell them the best stories to get you back onto the learning track, so to speak.

Rebecca: So you begin your book by discussing a variety of widely believed learning myths. Can you talk about a couple of these myths?

John D.: Sure, one of the biggest and I really don’t think this holds true for college students, but K through 12, definitely, it’s just the myth of learning styles. So, we all have learning preferences. I certainly have preferences. I wish my instructors would sing to me all the time and do my learning through music. It’s just that’s not going to help me learn calculus, and so forth, where really hard work needs to be done. So it’s natural to accept a myth like that, because it makes everyone feel good. It really can also undermine learning because it makes us want to do things that are more aligned with our preferences, but sometimes misaligned with most effective practices. So that’s kind of one learning myth, which I hope, in 10 years, if we were to have this discussion again, it would not come up because it was completely dispelled. But there are a variety of others too, that I think can undermine student success too.

Regan: And I think something else, that we really tried to put ourselves in students’ shoes, and we built on what our students said to us. And another one I wanted to highlight in response to that question is so often I think our students think that learning is up to us, if they have a good teacher they learn. If they have a bad teacher, they won’t learn. And undoubtedly, teacher effectiveness is important, but it’s also up to the student. The student has to do something, the student has to take accountability. And early on in chapter one, we show that at least half of the variance half of the accountability for learning is in the student hands. Teachers are also responsible, but really, if a student isn’t doing their bit, then learning is not going to happen. And now once you are ready to do your bit, well the rest of the book tells you what to do. But you’ve got to realize that don’t just sit back and go “Come on teach. Do your magic.” We all like doing our magic. But we can do all the magic in the world if you are not joining us in this adventure, and I’m thinking adventure a lot, because Zelda Stairs of the Kingdom just got released recently, and my household seems to be quite into it. So yeah, join us on the adventure.

John D.: Absolutely. Let me just jump in with one other learning myth that really gets me, and it literally gets me, because I fall prey to the same thing. That’s if you’re struggling, you’re not learning because it’s frustrating. And I often try to pick up new skills, I’m teaching myself to play the guitar now. And I struggle, and I get frustrated, I want to throw my guitar down because things aren’t happening. And I have to remind myself that sometimes it’s the struggling when you’re doing the most learning. And if we could allow students to realize that if you’re not struggling, you actually may not be learning, and that you should embrace the challenges, the hurdles that are in front of you. And it turns out, some of those hurdles are embracing the most effective learning techniques, which produce struggles in learning, but yet also produce the best long-term outcomes. Whereas other ways to prepare for classes and so forth, the way I’d like to teach myself to play guitar, makes it seem very easy. But then there’s really no progress being made. And if there’s any method I could really want to undermine, that would be a major one, that sometimes struggling is a good thing, not all the time, and it’s up to the students and the instructors to figure out is this a good struggle or bad struggle and how to correct that.

John: Those times when we try to give students desirable difficulties, they often find those to be somewhat undesirable. And I often get comments in my course evaluations that “he’s making us learn it ourselves.” And my response is always “Well, I can’t really learn it for you, I can give you the tools, I can give you resources, I can help you learn this material, but ultimately, you have to do the learning.” Going back just a little bit to your comment, John, about learning styles, I had a discussion in an online class, which is mostly upper-level students. And about three quarters of the students in my class were very strongly convinced of that myth of learning styles. So it hasn’t gone away. We hope it ma, but it is very pervasive, certainly in K through 12, where they’re regularly testing it, but some of the students have mentioned that they’ve been tested here at the college in some of their classes. And it’s again, really frustrating to see that, but it’s pretty pervasive.

Rebecca: Yeah, I was struck too, John, when you were talking about both learning styles and the struggle, because as I was working with mostly graduate students this semester, I had those exact conversations with those students. But it may have been the first time that they’ve really come up with some barriers that they faced in learning, and really had to reassess how they were working towards their goals.

John D.: Absolutely, and nothing but utmost compassion for students and sympathy, because in many ways, the experiences they had before college allowed them to use ineffective strategies and still get by. So things didn’t seem like a struggle, the difficulty now, the first time, then you hit a struggle, when you really have to change strategies or techniques, you may not realize what to do. It might seem frustrating and overwhelming, when in fact, now’s the time when the real learning begins.

Rebecca: Yeah, and I think sometimes when the subject matters new, as well as the strategies are new, students can immediately feel a little cognitive overload. [LAUGHTER]

JOHN D. Absolutely.

John: And part of that I think may be related to how they’re often taught throughout their whole career. In general, much of the advice they get in elementary and secondary schools, and sometimes in college, encourages them to adopt strategies that may not be that effective. Often, people still use high-stakes exams. And people don’t do a lot of interleaved practice or spaced practice, which are things you talk about really nicely in your book. And the incentive structure for students is perhaps to favor things that align well with short-term recall without that longer term learning. And your book can help, if we can get students doing this, but it seems to me like we also have to do a bit more work and getting faculty to adopt some techniques that may encourage students to use techniques that are more effective. Might this be something we could build into our classes a little bit?

Regan: Actually the riff over a couple of different things you said, John, I think there are two things going on here. One is, a lot of the issues that we run into with our students are actually pretty easy to pinpoint. And that is high school. And when I say high school, I’m not saying “Oh, high school teachers,” I’m saying the whole high school environment. One of my favorite examples is the syllabus. Many of us college and university faculty complain that students don’t read the syllabus, and I had a great conversation with my 17 year old. I was talking about the syllabus and he said, “Yeah, the syllabus is useless.” I said, “Excuse me?” and he said, “Here’s what we do with the syllabus. We spend a whole day, a whole day, reading through every word on the syllabus, but the syllabus is not really designed to help us learn.” And I looked at his syllabus. And sure enough, the syllabi he had didn’t really map on to the best practices that many of us listening employ. And I could completely see why he had that whole. “Yeah, I’m not going to read the syllabus when I get to college. I said, “And of course, you will now,” but it was a great conversation where I really got to see the mindset. And I think that’s just one example, where a lot of this happened, where it’s like, let’s see, what have they just been used to? Then add on top of that, how learning changed during remote learning. That’s another whole mix to the whole issue. And then, add one more, the misinterpretation of, I think, learning styles. Before we started this call, we were joking about the fact that our book is now in an audio version. Now, because it’s an audio version, does it mean the auditory learners will be better than those….? No, it just means that sometimes when you want to listen to something that works well, other times reading something works better. So it’s not this whole “Oh, look, there’s this one modality and I’m a person of this one modality.” And I think that preference that John D. so nicely used… it’s preference, right? …and we have preferences. But let’s also not forget the take home message of that Pashler et. al 2008 study, which is the more styles that you use…. And here’s where it goes back to desirable difficulties, mixing it up, is actually better for learning, no matter what your preference may be.

Rebecca: Oh, but mixing it up is so uncomfortable. [LAUGHTER]

John D.: It makes things a lot more difficult, right? One advice I would give too, that my teaching has changed dramatically, as I’ve gotten more into learning sciences. And at least for someone who teaches, I hate to say this, a non-essential course course that I love and courses that I love for undergrads, I realized I was teaching them too much. So I put them into almost a defensive mode, where the only way to prepare, because they were overwhelmed, is to cram as much as they could and hope to pass the exam because it was a tsunami of information. So over the years, I’ve followed the model, less is more, it’s up to me as the instructor to figure out what is really essential, and how to give them just as little as possible to make their experience as large as possible. So that they then can use the techniques to learn the content well enough so that they can keep that content with them a lot longer period of time. So it’s really kind of changing styles. Now, don’t get me wrong, if I’m teaching chemistry to pre meds, and you need to know chemistry, then you need a different approach, because there’s a lot to learn, it’s difficult, and aspects of our book really focus on how to meet those challenges too. And I think students, just like instructors, need to pick and choose where they want their battles, you don’t have to ace everything as a student, but decide what you really want to do well at, and then use the most effective practices to nail it as you work forward toward a long career and lifelong learning.

John: One of the things you include early in your book is a list of what research finds to be effective and what research finds to be ineffective, side by side, which for faculty use could be really helpful except for some degree of similarity between those two lists. Could you just talk a little bit about that.

John D.: I like to think about it, as much as I’ve not talked about it like this, as more or less effective versus effective and ineffective. And I’m not gonna like reference to that table necessarily, but let me talk about one that I badmouth a lot, which is highlighting as a learning strategy. And like Sharpie has not contacted me with a lawsuit or something because it sounds like I don’t want people to have highlighters. And what we mean by less effective as a learning strategy here is, is that using a highlighter is just the beginning of a learning adventure and not the end of it. So there are great uses of a highlighter of rereading material, of these things that really don’t lead to a great deal of learning, because they kind of are the stepping off point or the catapult for learning, so to speak. So is using a highlighter ineffective for learning? Absolutely, it doesn’t help you learn the content that well. But what I recommend to students: to highlight everything they want to learn as kind of a different approach to that strategy, so that they then apply the most effective strategies in learning it. Yes. So as much as we kind of play off ineffective to effective, it’s more like every strategy in its right place. And some strategies that can be generally ineffective for very specific uses might be relatively good, and what I think Regan and I try to do, because it’s way too much to think about if you’re not a learning scientist, like one exactly show us the specialty strategy, but we try to focus on are just strategies that students could learn just more generally to do well in any particular content. So kind of low hanging fruit, so to speak, where you don’t have to be a learning scientist to make decisions about which specialty strategy to use in every place. But here’s some kind of fail-proof strategies that can help you anywhere.

Regan: I want to actually say there’s that flipside too and John, this is something you asked about earlier when you talked about teachers changing what they do, and I think John D, and I joke about the companion volume to this is Teach Like a Champion. But I think the nice thing about that table of high utility and low utility strategies is that it’s not just the student who needs to be aware of the fact that some are high and low. But I think we take it further to say why are some of those high and low, and then here’s the key companion piece, which is, we hope that instructors reading this can also take those tips, and be ready to share different options with the students. Because I know both John and I have compared notes on this, when a student comes to our office after an exam who’s not thrilled with their score, who has not scored well, we both do some pretty intensive. “Alright, so tell me exactly what you do.” Just that conversation is so important. We both ask to see their notes. And of course, every once in a while a student says I don’t take notes. Well, let’s start right there. And that’s why we have a whole chapter on note taking, and why take notes and how to take notes. But I think that’s exactly it, is so many of us. and I’ll say us, not just our students, so many of us are so used to doing things just one way that we’re not ready. And we don’t look enough about is this really working or not. And I think that’s what we really push with a lot of very real examples ripped from the headlines of our lives, as it were, where we talk about, look, this is what somebody’s tried, and here’s what they did wrong. And here’s how they can make it better.

Rebecca: I wanted to talk a little bit about that chapter on notetaking. Faculty often make the observation that students don’t take effective notes, but then don’t necessarily respond by teaching students how to take notes or giving them some strategies to improve that skill. Can you talk a little bit about some of the things that finally might show students or offer students to take more effective notes?

John D.: Absolutely. So the one thing that I would recommend highly is that if we all agree that taking notes in its own right is not where the learning occurs. And in fact, research comparing people who take notes versus who don’t take notes typically showed no differences between the two. Now, the key is, when you look at these studies carefully, you’ll realize the folks who took the notes, and who didn’t take notes do equally poorly. So it’s not like they do equally well. So the notes are a artifact that in many cases students can use to really learn the content well. So the question is, how do you get all the correct and appropriate information into those notes? So one thing that teachers can do, that research shows really helps well, it’s just flag important content, literally tell students “Okay, time to take notes, this is critical, I need you to know this.” One of my favorite statements, ”The next thing I say, will be on the exam.” [LAUGHTER] And see how many people start taking notes. Now, of course, I would love to see notes that are embellished with examples, with questions, and so forth. But at least getting the rudimentary content down there as an artifact, so students have it to utilize as they’re preparing is so important, or having teams of students take notes and share those things as well is also good. So I think for teachers to be compassionate, but also to help your students sometimes identify what’s most important, by speaking louder by saying, “Hey, this is really critical.” Sometimes I have to even flag it to not only just critical, but it’s going to be an exam, you’d think those two things meant the same thing, but sometimes students need a little help. I think it’s important, and there’s nothing wrong with telegraphing what is most important in your class. In fact, let me restate that. There’s everything not right about not telegraphing. Now, try to parse that. But what I’m saying is, we should be telling the students what they need to learn, how we’re going to test them on it, what we’re going to test on, and allow them to meet those successes. And I think good note taking, and helping students take notes by telegraphing what should be in there, can be very useful, in my own experience, and at least some of the research too.

Regan: Notice what else is in there. If we do want students to take good notes, we don’t want to be talking extremely fast. We want to make sure we have pauses, we want to make sure our slides, to the extent we use them, are not so packed with information that either they want to copy it all down or they get it all as a handout so they don’t have to, but there are all these nuances. And I want to go right back to that finding about the students who take notes and the students who don’t take notes. The bigger issue is what are you doing with those notes? I think far often students just think about it as “I’m going to record what the instructor said.” Well, you’ve got to go back and revise them. You’ve got to go back and check and see if your understanding is right. You need to review them. The issue is, I think, when people take notes, they’re not doing enough with those notes. And there are ways where we can talk about how to take better notes and they’re not. When I teach 100, or intro level classes, every once in a while, I will stop, and I do this during the first week, I will stop. And I’ll say, “Alright, here’s a reality check. If you took good notes, you should have the following. How many of you have it in your notes?” And I do that a couple of different times that first week or the first two weeks to get into that habit of you should be taking notes, you should be doing something about it. I also want to say something that didn’t make it into that chapter on note taking, but it’s gone up another shot, because I’ve seen this so often, in the last year and a half or two years, students taking photographs of the slides. Reality check, people, that’s not taking notes, taking a photograph of a slide is not taking notes. But I think in some students’ minds they’ve taken notes, just more effectively. And there are studies now rolling out that show that the photograph slides versus note taking versus controlled no notes. that the photographs and no notetaking is doing the worst on quizzes. But I think that’s something else we’ve got to take up” how you take notes is important and taking a photograph is not it.

John D.: Let me spin back to something that Regan said that I think is great, too, as far as helping students develop better notes, I begin, and I know Regan does this as well, every class with a no-stakes quiz. And I use that for a variety of different reasons, some of which we talk about in our book, but one of them is first try to answer the quiz multiple choice question without the alternatives from memory. And if you got it from memory, that’s important, because that’s going to help you learn. And then if you don’t get it from memory, now see if you can answer using your notes from the last class. And if they can’t answer the question using their notes, we have issues. So they’re not taking complete notes, they don’t understand their notes, or the issue could be with me. Occasionally, I look at the notes and realize, “Oh, my goodness, I must have said something wrong last class, because everybody’s notes are incorrect.” [LAUGHTER] And if everyone’s notes are incorrect, the source of the error is probably me. So it’s a way to help students understand that the notes are the vehicle to understand the content that we really want them to learn. And there’s ways to reinforce that through no-stakes quizzes, and so forth to help them understand that what we’re doing in the classroom is providing all the scaffolding to help them succeed.

John: One of the things you emphasize early in the book, and throughout the book is the importance of students developing their metacognition as well as their self-regulation skills. Could you talk a little bit about why that’s important, because I think that’s something that’s not emphasized enough throughout their educational experience,

John D.: I’m gonna just focus on one aspect of our particular viewpoint on self regulation, and then I’ll let Regan go with the more fun stuff. I want students to really succeed in school and in college. But quite frankly, as a college educator, I want students to succeed in life after they leave here. And one of the most important skills that anyone can learn is time management, that’s about self regulation. And believe it or not, I have actually students tell me, it’s like, “Well, I know, managing my time and developing plans, that’s just not my cup of tea.” (speaking of teaching for tea). But it absolutely should be, because effective people develop goals, plan on how to get them, and manage their time to get there. So if students can take something from this book that I think is highly valuable about self regulation is to look at our examples and our encouragement and inspiration to manage your time, make goals and plans, and develop a time management strategy that works well for you. Because that’s ultimately going to lay the foundations for a successful use of other strategies we discuss in the book, but also for a highly successful life, whether it be success in your job, your hobbies, raising a family, or whatever, successful people manage their time and self regulate. That’s something that we really do push in the book, and some of it’s not necessarily metacognition, per se, but it’s about really taking ownership of your life and deciding how to get things done effectively. So time management, for me is so important aspect of this book.

Regan: I think whenever you think about self regulation, I really try to stress two elements of it on one hand, especially think about metacognition is the: “What do you know? Do you know this stuff?” And there’s that whole “Am I ready for this exam? Do I really understand this concept? Do I know how to do this?” That’s the classic metacognition, but even more broadly, when we talk about plan and monitor and assess your knowledge, that’s, I think, really important as well. And of course, the tips such as retrieval practice is great for the assessment too and to assess yourself. But I think I love the way John talks about time management because when we talk about self regulation, a big issue that I hear from students a lot is where those two things interact, not being able to regulate how they use their time, especially when it comes to social media. And students will say to me, “Yep, I start scrolling Tik Tok, and before I know it’s an hour later. Well, that’s a basic self-regulation issue. That’s a time management issue. That’s a planning issue. And I think in our planning chapter, we recognize this and we tell students “Look, it’s okay to enjoy social media. But guess what? Plan your social media timr. So it’s not like you’re saying, I’m not going to use social media. Be real. If that’s what you enjoy, sure, look at some social media, but plan it out, allocate time for your social media, just like you allocate time for your work so that everything fits into your schedule.”

Rebecca: Allocating time, it feels like have a lot of conversations with students about that on a regular basis.

John D.: Oh yeah.

John: Are there other topics you’d like to emphasize about your book that we haven’t touched on yet?

Regan: Well, I think one thing that if you’ll notice, the last chapter in our book is all about things that, for the longest time, nobody thought related to studying: sleeping, eating, physical activity. But I think that is just so important. That is so important. And gosh, I know, I don’t think I ever shared this with John. But sometimes I almost played with putting that chapter first. We did put it at the end, because I think it’s really important. But I think most students don’t realize how those life things interact. And I think for those of us who read up on higher education, there’s just so much written about burnout among students, among faculty, and all of that relates to those things we talk about in the last chapter. It’s prioritizing sleep and prioritizing good eating and prioritizing physical activity, prioritizing just getting out and getting some air. And some of my favorite comments that I get back from students… it happens a lot when I teach health psych… but even otherwise, where they will say, “you know, yeah, I caught myself scrolling, I went outside for a walk, and it was just five minutes, but I came back and I could tackle things better.” And I think, people, it’s 360 degrees of living is what we’ve got to practice.

John D.: And I love that Regan. Because obviously, we saved the best for last with that chapter. But, come on, to do all these things. It’s a time-management issue. If I’m going to work, all of this in, how can I do so so I have enough time for play and I have enough time for work, which are two important aspects of life. So, totally agree.

Rebecca: Well, they always wrap up by asking: what’s next? [LAUGHTER]

John D.: Wow, that’s a big question. And what’s next, I would say for me, the strategies that I think are most effective, that work the best, unfortunately, have also been investigated the least, because they’re difficult to investigate, they involve multiple sessions for students to engage the same material, and so forth. So as someone who was kind of born and raised in a laboratory, either my own lab or laboratory of the classroom, what I’d love to see is just much more evidence-based research focused on further understanding, not only what works best, but how students can engage in the best practice in the most efficient way. So it’s balancing both an understanding of wanting students to obtain their learning objectives, but also understanding that they want to do so effectively and efficiently. Because there’s so many aspects to life beyond just school. So I’m always going to say more research, although I could totally understand some would say more application too, but I’m gonna go what’s next is some more research and all these really effective techniques and how to use them more effectively.

Regan: And it’s that last part, John, that is what’s exciting me right now, I’ve been learning a lot more about the whole field of implementation science, which is just because something is effective, and something is efficacious, what are the factors that influence its implementation? …because whether it’s a student reading our book and trying to implement our recommendations, or a faculty member, instructor, teacher listening to us and trying to implement what we’re saying, there are still nuances, there are still contextual factors. And I think we’re just getting better at intentionally and systematically teasing apart what some of those issues with implementation are. And for me, that’s been pretty exciting. I was fortunate to read a lot of a fellow Oregon State University, new PhD, Dr. Rachel Schweitzer, who’s done a lot of this work on implementation science, and some of her ideas. I’d love to be able to test them in my class, to dovetail with continuing research on study techniques and how to make it happen more,

Rebecca: Some really important stuff that needs to happen, for sure.

John: And thank you for this book and all the other research and work that you’ve done in support of improving student learning.

John D.: Thanks for having us on.

Regan: Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

290. Transparency in Learning and Teaching

While instructors know what they expect from students, these expectations are not always clear to their students. In this episode, Mary-Ann Winkelmes joins us to discuss what happens when instructors make their expectations transparent to their students.  Mary-Ann has served in leadership roles at campus teaching centers at Harvard, the University of Chicago, the University of Illinois, the University of Nevada – Las Vegas, and Brandeis University and is the Founder and Director of TILTHigherEd.

Transcript

John:While instructors know what they expect from students, these expectations are not always clear to their students. In this episode, we explore what happens when instructors make their expectations transparent to their students.

[MUSIC]

John:Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John:…and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Mary-Ann Winkelmes. She has served in leadership roles at campus teaching centers at Harvard, the University of Chicago, the University of Illinois, the University of Nevada – Las Vegas, and Brandeis University and is the Founder and Director of TILTHigherEd. TILT is an acronym for Transparency in Learning and Teaching in Higher Ed. We are very much fans of the TILT approach and have referred to it often in workshops on our campus (and on previous podcast episodes). Welcome, Mary-Ann.

Mary-Ann: Thank you. I’m really delighted to be here with you. And I appreciate the opportunity to speak with you on Tea for Teaching.

John:We’re very happy to have you here. You’ve long been on the list of people we’ve wanted to invite. So we’re very pleased that you’re here today. Today’s teas are:… Mary-Ann, are you drinking tea?

Mary-Ann: I am indeed. And I’m drinking a Sencha green tea today. That’s my new favorite kind of green tea, Sencha.

Rebecca: Nice. I have English breakfast today.

John:And I am drinking a mixed berry Twinings black tea…

Rebecca: Hmmm.

John:…which I haven’t had in a long time. I wanted to mix it up a little bit today.

Rebecca: …mixing it up with mixed berries. So, Mary-Ann, can you tell us a little bit about how the TILT project came about?

Mary-Ann: Sure. This was years back, I want to say in the early 2000s, late 1990s, where I was working at the BOK Center for Teaching and Learning at Harvard University. And I was leading a seminar group discussions about teaching and learning. And we began to think about the question: “What happens when you tell students why you’re teaching how you’re teaching, just what happens when you tell the students more about your choices as an instructor, how you’re choosing to shape the learning experiences for the students?” And that’s not often something that we think about first when we’re thinking about what’s the content of the course. But we began to think about that a lot. And we had a kind of metaphor about the Wizard of Oz, and pulling back the curtain to show what was happening behind the scenes to build the experience. And then somehow through that conversation, the word transparency emerged. And that became the word that we used pretty regularly from that time on. When I moved to the University of Chicago, that was the word we were using, and it kind of stuck. So that’s kind of where it started. And it started alongside of my career as an educational developer. And it’s kind of been, for me, in the background or on the side, as something that I’ve been kind of tracking along with as a project. It’s still there, it keeps going. And just about a year ago, I began to work on TILT as my full-time job, which I’m really happy to be doing now because it gives me an opportunity, not just to do a guest talk here or there, or a keynote address, which is usually a one time-interaction. But now I have the flexibility to connect with institutions around a longer-term project. So if there’s a faculty learning community that emerges from a first talk that I would give, I get to follow up with them later and see what’s happening and check in with them. Sometimes I get to see the assignments before and after, which I really like. And I invite those now, because we’d like to publish some of those on the TILThighered.com website. And there are some schools that I’ve been working with in the state of Washington for several years now running with their TILT projects. And that emerged from a project we did with the entire state system of Community and Technical Colleges in Washington State. So I have opportunities now like that, where I can work with larger scale TILT projects that take more time, because this is my full-time job now. And I’m really happy about how that’s working, because I feel like it’s getting larger beneficial impact for students in a way that’s more efficient than when my full-time job was at an individual institution.

John:Could you give us an overview of the TILT framework?

Mary-Ann: Absolutely. So the TILT framework is meant to be a very simple tool that is a framework for an ongoing kind of communication among teachers and students. And in all of our studies, we asked teachers to use this framework in their own way at their own discretion, because we know that it’s not really possible to expect that people would do the exact same things with it. So our research is based on the premise that people are using this framework in their own way, at their own discretion, in a way that feels consistent with their teaching style. So there are three parts to this framework: purpose, task, and criteria. And what we ask in all of our studies is for teachers to engage students in conversation about three aspects of a particular assignment or a project or even an in-class activity. Before the students do a piece of work that we want them to complete, we’re asking for teachers and students to have a conversation about three aspects of the work before the students start working on it. And those three aspects are the purpose, the task and the criteria. Now the purpose kind of consists of two pieces. The first part is talking about the skills that students will practice while they’re working on the assignment. And then how are those skills useful, not just now in this course, or maybe in college and other courses, but how are these lifelong learning skills that will be useful for the student in their careers after college or in their lives ongoing? And then the second part of the purpose is about the content knowledge. What new information or what disciplinary information will the students be researching, or gaining, or applying when they’re working on the assignment? And how will that be also similarly useful to them, not just now, or in college, but beyond in their lives? The task, that’s the second part of the TILT framework, and the task is sort of about what are the teacher’s expectations about how students will approach the work? And for the students, it’s kind of like mapping out their game plan, like, what’s the first thing they will do? Will they Google something? Will they go to office hours? Will they go seek out a research librarian? Will they go into the lab and start mixing something like, what’s the first thing they’ll do? And then a sequence of what they plan to do after that until they submit the work. In an ideal world, the teachers and the students would have similar expectations about how that would go. In some cases, though, teachers have a pretty legitimate pedagogical reason for hiding that, that they don’t want students to know how to do the task. And I found this to be the case, particularly in fields where creativity is really important: performing arts, studio arts, even engineering or some STEM courses, where teachers really want students to cast about for a while and kind of use their imagination and see if they can come up with something unique, if not into the discipline, at least unique for the student to try to figure out some new process. And there’s value in that. When teachers want to do that, we did have some pushback from teachers in our original TILT research studies, where they said, “What happens if we don’t want to tell students how to do the work, like part of the task is for them to figure out how to do the work?” So in that case, we asked for those teachers to just say something like, “Part of the purpose of this assignment, in addition to the skills and the knowledge we’ve talked about, part of the purpose is for you to struggle and feel confused, while you invent your own approach to the question.” And we think this is what helps to preserve the student’s sense of confidence and their sense of belonging. Because instead of having that moment of panic of “Oh, no, I don’t actually know how to do this, I don’t even know where to start, I don’t know where the resources are, I don’t know what I’m doing. Maybe I shouldn’t be doing this, maybe this isn’t for me, maybe I shouldn’t be in this major, or in this course.” Instead of going to blaming it on themself or to questioning whether they’re up to the task at all, students can say instead, “I am totally lost right now. And that is exactly where I’m supposed to be. I know I’m on track. I’m doing great. This is the confusion part that comes before the clarity. And I know that because we talked about that and the professor said, this is part of what we expect to happen. This is intentional, this confusion, you’re supposed to feel lost right now.” So that’s sort of what we can say about the task. And the benefit of students knowing upfront what the task is, or knowing how they plan to approach the assignment or the project, the benefit there is that students get to spend 100% of the time they’ve allocated to work on this project, doing their best quality work, and they don’t lose time trying different approaches to see if this or that is going to work or looking for resources that aren’t what the teacher intended for them to be using. Instead of losing time, on the “how,” students get to spend their time so that what teachers receive then is most of the time what we’re looking for, which is “What is the student’s highest capacity right now?” Let’s see an example of the best work that the student can do right now, so we know where they’re at and we can bring them further so that they can advance in their learning. But if we accidentally end up in a situation where a teacher didn’t intend for the students to be confused, they expected the students to take a particular approach that they may have even mentioned at some point in class. So that’s why they think the students know that that’s the expected approach. I don’t want to say the correct approach but at least what they expect students to do. So if we think that students know how to do what we expect them to do, and the students don’t know what we’re expecting them to do, then there’s this chunk of lost time, where what we’re measuring then in the end is what happens after the students spend a chunk of their time lost trying to figure out how to approach the work, and then whatever time is left after that doing their best quality work in the amount of limited time that’s left. So part of the “task” piece of the framework is about what do we want to measure? Right? Do we want to be assessing the best quality work that students can do? Or do we want to be assessing what happens when you give a really varied, diverse group of students a particular assignment to do and you don’t give them 100% clarity about how to do it, and then kind of what you’re measuring is which students have, through no fault of their own, not encountered that information in their lives before coming to this course. And then you also get to identify who are the students that maybe because they had some other kinds of privileges that not all the students had, who are the students that can figure it out faster, because they come equipped with those privileges. So you can begin to see that this is an equity issue. So if talking about the purpose of the assignment kind of speaks to the student’s motivation, and to the value that they will gain from doing the work, and maybe to their ability to assess if they’re getting that value while they’re doing the work, the task speaks to even more of an equity situation where we’re trying to get all of the students to the same starting line of understanding of how to do it, and of having all the resources they would need to do the work to complete the work. And we want to make sure that students are all at that same starting line before they start the assignment. So that’s kind of the equity piece of this. And then finally, the third part of the framework is about criteria. We want students to be able to understand while they’re doing the work, how well are they doing. We want them to be able to make corrections, if they end up with a finished version that doesn’t look like what successful work would look like in this kind of a scenario. But if the students have never seen what successful work looks like, and they probably haven’t, because why would you assign them to do something that they’ve already seen many examples of; they wouldn’t be learning anything new. So kind of by definition, students aren’t going to know what successful work looks like when it meets this or that criterion in the discipline. So what we encourage teachers and students to do there when they’re considering the criteria is to offer students more than just a checklist or a rubric, because the words on a rubric or checklist might mean something different to the student who hasn’t done this kind of work than they do to the teacher who’s really immersed in this kind of work. An example I sometimes offer is, let’s say, I asked students to write up an analysis of a 15th century wooden painted sculpture of the Madonna and child from when I was teaching Italian Renaissance art history courses. In an art history course, the word analyze, like the tasks, the actions that you take when you are analyzing something, that’s a very, very different activity than analyze in the context of an economics course, or in the context of a chemistry course. But if the student hasn’t done this kind of analysis before, you can’t know for sure that they know what you’re asking them to do. So we kind of have to talk that through and students are going to need to see some examples of real world work in the discipline so that they can, with you, in a class meeting, talk about how do we evaluate analysis in this example from the real world, or in that example from the real world. And you won’t find any one example that matches every criterion of the assignment you’re asking students to do, usually, so you need several examples. The benefit of several examples is also that you can begin to talk about the relative success with which different examples are meeting a particular criterion as well. So once we’re in a conversation with students, and we hear back from them, that they’re telling us, what we had hoped they would understand about the skills they’ll practice and the knowledge they’ll gain, that purpose, about how they’re going to approach the work, the task, and about how they’ll know that they’re doing good quality work, the criteria, once we hear students telling us that, that’s the moment that things have become transparent. It is that activity of communication, that conversation with students about purposes, tasks, and criteria, that’s where the transparency comes from. And when we are done with that conversation, we know that students are at the same starting line of readiness In terms of their understanding of what they’re going to do, and also, in terms of their confidence that everyone has the resources that they need, in order to complete that work

Rebecca: What faculty believe is important for students to learn doesn’t always align with the goals of students. Can you talk a little bit about some strategies for bringing these into better alignment?

Mary-Ann: Sure, I think that this kind of speaks to the purpose part of the transparency framework. And often teachers are expecting students to learn something that is very valuable, we wouldn’t spend our time teaching things that don’t have a lot of benefit for students or that they would only use today and it wouldn’t be useful to them later in life. We like to teach things that have value. And so, when we are communicating with students about that value, we’re talking about the skills that students will be practicing. They won’t perfect them on this assignment, but they will begin to strengthen a particular kind of skill set. And they will gain some sort of disciplinary knowledge that can be useful to them later. And we know that sometimes disciplinary knowledge changes over the years as people discover new things and publish new things in any field. Sometimes that knowledge changes. But having some knowledge now does give you important value if you’re going to continue in that discipline or if you want to understand basic principles of a discipline that you might find useful elsewhere. So if students and teachers have a transparent conversation or communication, it could be a written communication, it could be something that they record and put on a website, it could be an asynchronous kind of conversation in an online course. But whatever form that communication takes, I think students and teachers when they’re on the same page about what the knowledge is, what the skills are, that are the focus of this assignment, students will feel more motivated to do the work, because they’ll see that it has benefit for them. And it doesn’t feel like a rote exercise, or just churning out another problem set or another art history analysis paper. There’s some value here that the students know upfront what that value is. And when the teachers hear the students reflecting back to them in this communication, that this is the value that they will be gaining, then we know that students have a kind of motivation to benefit from this assignment.

John:One other issue is that students have come up with some way of learning while they’ve been in elementary and secondary school. But those methods that they picked up are not generally the ones that are most effective. How can we encourage students to adopt learning strategies that they may be resistant to because for example, students, when there have been surveys of what types of learning strategies they found most productive, students often say they prefer to be lectured at, because they learn more from the professor that way. And also, many students don’t like active learning strategies. While they learn more, they don’t perceive it that way. Partly because of those desirable difficulties you referred to before, that when they’re struggling with something, it’s a little bit less pleasant than sitting there nodding and smiling and having everything seem to make sense. How can we encourage students to accept those desirable difficulties associated with learning so that they can learn more effectively,

Mary-Ann: I want to say that this is something that the TILT framework can definitely help us with. And this is not an uncommon phenomenon at all, I even find in my TILT workshops that I do with instructors, that instructors don’t love collaborative learning either. And in fact, many of these TILT workshops that I do will begin with some kind of a research review about “How do we know TILT works? What are the studies and what do they tell us and show us the data?” So we get off on this kind of role, where we’re almost in a traditional lecture format, where like someone’s delivering some information, and people are listening, and then they have questions about it. Or maybe they have challenges to say, “Wait, this doesn’t make sense, let’s talk about this.” And then I kind of switch the method that we’re using. And I’ll ask people to break off into small groups and begin to analyze a particular assignment and talk about where do they see the purposes, the tasks, and the criteria? Before I do that, I acknowledge the fact that we are shifting gears, and that we were doing fine with this sort of Q&A format. You know, look at the research and then think about it and talk about it. Ask questions. Why would I switch that up now? Like we were on a roll, we were doing great. Everybody was sort of on board. Why would I change that now? And so I use the TILT framework to talk about why we’re shifting gears now. What is my purpose in having you use this different method? So if it’s a peer learning method, as it is in the workshops, or as it might be with students in a class, we want to tell students: “Why are we now manipulating your learning experiences this way? Why would I do that to you when I know that sometimes students resist this, when I know that it can be uncomfortable, because I don’t personally always like to do it when I’m in a learning experience?” So if we can tell students, here’s why this is going to benefit you, because you don’t just hear it, but you have to struggle to apply it, you have to fit it not to the situation that I was talking about, where it all sort of makes sense when it rolls over you and you’re hearing it. But you now have to take the principle of what we were talking about, and apply it to this new unfamiliar scenario. And the benefit of that is that you will discover you will hit a barrier at some point in that process, where you will discover the exact piece of information that’s missing for you. You will discover exactly where you hit a barrier to your understanding. And you will have an opportunity right now, right here with me, the teacher in this class, to address that confusing point. And the benefit of doing that now, as opposed to later when you’re doing a graded assignment, is pretty obvious, you get the benefit of having the difficult learning experience in a safe environment that doesn’t lose you any sort of points on your grade. It doesn’t have any negative impact on you the way that it might if you waited until the end of the term to do some massive project and you hadn’t really done a lot of the homework or done a lot of the practices and so you didn’t really know what you didn’t understand until it was kind of too late to do anything about. So I think in short, what I’m trying to say is when we’re asking students to do something uncomfortable, that has a really solid pedagogical reason, that has evidence behind that, it is an evidence-based practice, we want students to know that upfront, because that then will increase their motivation to do it, because they see how they’re going to benefit if they do this thing.

Rebecca: One of the things that students often struggle with is when they start new courses with new faculty, and new ways of doing things and determining what the instructor will expect out of them and out of that learning experience. Can you talk a little bit about how the TILT framework could allow students to shift their focus to learning if it was adopted in the design of the course rather than just an individual single assignments?

Mary-Ann: Yes. And in fact, this is a way that lots of faculty are using the TILT framework, is to think about how do I TILT not just a single assignment, but a whole course. So usually, when people are introduced to the TILT framework, the original ask for all our research studies is would you please do this two times in an academic term, just twice? Because we wanted to see how little change could you make and have a beneficial impact on students’ learning, because small change is much more likely to happen than massive change. But once you’ve made that small change as an instructor, and you see that when you do this with two assignments, there’s some real benefit for students. And on the TILThighered.com website, there are publications by faculty who talk about not just how the quality of students’ work increases, but how the teachers experience in grading, or in responding to students, or in how many students will ask for an extension at the last minute, like these difficulties that teachers often face are diminished, while the benefits for students and the quality of students work increases. So once you begin to see this in the small scale of assignments, teachers then, maybe in the subsequent term, will think about what else could I TILT? Could I TILT in-class activities? Could I TILT a unit of this course? Could I TILT the whole course? And then the effects or the applications can grow. So we can apply this to a single assignment, we could TILT a whole course, we could TILT a curriculum in a department, we could TILT a program, we could TILT an institution’s learning outcomes and thread them through not just all the courses, but through all the co-curriculars too so that students might discover in their work-study job that they’re practicing one of the critical thinking outcomes, that’s a goal for the whole university that connects with what they were doing in their accounting class. And then we can even think about this in terms of a national framework of learning outcomes as well. So there are many scales at which you can apply that to a framework. And one of the things that I’m really enjoying about doing TILT full time, is that I can work with groups of schools, groups of institutions, so not just the Washington State group that I mentioned to you, but several weeks ago I was in the state of Kentucky where working with teams of teachers from institutions across the state, for the whole state system, to think about aspects of how do you map out a path for students to succeed in fulfilling their curriculum? And then how do you pursue that path? How do you complete that path? And in that case, we were using the TILT framework as a strategic planning framework to think about once we know what the plan is, like, once we’ve mapped out our plan for how students can effectively complete their degrees, how do we then communicate the value of that degree, not just to the students who are doing the degree, not just to the students’ families who may be contributing to the costs of doing that degree, not just the costs of the student’s tuition, but the cost of the student not being an earner in that family. And we want to communicate this to all the stakeholders, so the students, their parents, faculty, and staff at the institution, to state legislators who may be voting on packages of funding to higher education in their state, to individual grantors who might be funding particular scholarships. And we want to be able to communicate the value of this degree to every stakeholder in a state system that way. And the TILT framework is very helpful for thinking across multiple audiences, because that’s a pretty difficult task to communicate clearly to all of those different kinds of audiences. But it’s pretty essential for the success of higher education in this country. And so we spent a couple of days using the TILT framework as a strategic planning framework to think about how do you communicate the value of a degree? There are lots of ways that you can apply the TILT framework. Another example is I was working with a school in Texas over the summer, and they were TILTing their entire college success course. Many institutions have that kind of course in the first year, and some of them had TILTed individual assignments. And they decided they wanted to put the team of all the teachers together, and then subdivide that so that a smaller team of teachers was working on each week of the course. And then all the assignments and the lectures or discussions that would go into that week. And then we use the TILT framework as a larger framework to connect that whole course. So that from week to week, the purposes, tasks, and criteria were pretty clear. And students understood the path for all of their learning across that course.

John:Have you tried taking on the Florida Legislature? [LAUGHTER]

Mary-Ann: I have not.

John:That’s a real challenge, I suspect.

Mary-Ann: Yeah, I have worked with schools in Wisconsin. Last week, I was working with a school in Tennessee, right after a couple of their legislators were expelled temporarily. This kind of a framework, I think, can be effective in a lot of different higher education systems and contexts. That’s one of the beauties of it. Because this is something that teachers can do, starting right now, to complement any kind of larger, institutionally driven or federally funded program that might focus on student success. A lot of the time, those programs don’t necessarily feel like they’re directly connected to what faculty members are doing in the day to day in their classes. But using this TILT framework is something that you can do that will advance students’ success that will then make you feel more like you’re connected to these larger ongoing efforts that might be focusing on something that you don’t do directly, like targeted scholarship funding, for example. But that’s part of the beauty of the TILT framework is that it can work in many, many different contexts, and across different scale sizes of projects, as well.

John:And it works nicely for faculty because you end up getting work of the quality and the type that you expect, rather than getting student work that you find disappointing. And similarly, students end up doing work that they’re much more happy with, because they were not guessing at what the instructors want. So it just seems really, really logical. But it’s not always so widely practiced. Your efforts are really helpful for all of this.

Mary-Ann: I think one of the reasons why people might be hesitant to use the TILT framework, you don’t necessarily want to try doing something different that could suck up time that could take time away from delivering important content in the course, and what teachers have discovered and written about and published in the National Teaching and Learning Forum and other places you can see on the TILThighered.com website, what teachers have discovered is that if you take some class time to talk about the purposes, tasks, and criteria for a project before students do it, by the time that practice is completed, everyone has saved time; that time gets recouped, and students have learned a larger quantity of what we had hoped they we’d learn because when we deliver content in a course, we don’t know that students are absorbing it the way that we’d hoped or that they could apply it the way that we’d hoped. So I think by the end of the course, if you’ve used the TILT framework a couple of times, you’re in a situation where you’ve worked in a way that is more time efficient, somewhat, and you arrive at a place that, as you say, is more satisfying for students and teachers, because more of the time has been spent with the students doing the highest quality work possible.

Rebecca: I think one of the things that can be challenging for faculty initially is that if you’ve never communicated in this way, it’s hard to do it the first time, because anything you do the first time is difficult. But once you have a little practice doing it, it’s easy to adopt and expand across a course or across a set of courses.

Mary-Ann: That’s so true. And I think that the way that we’ve structured the TILT framework, it looks so simple, it’s a three-part framework. Applying it then gets you into some complexities that are important to clarify. I think you’re absolutely right, the first time we try anything that’s unfamiliar, just like for students, it’s more difficult. And then we kind of get the hang of it. And then it comes much smoother, and much easier. The TILT framework for starters, is pretty simple. It’s got three parts, right? And I think you could probably share a link to the one-page version of the framework that we give to students, that sort of spells out the framework: purpose, tasks, criteria, the knowledge and the skills. And then at the bottom, there are some of the evidence behind why we know this works and some footnotes, so that students can see on one page, this is a real thing. It works, it helps you. It is, in some cases, equitable, and it is probably worth giving it a try. And if you can see all that on one page as a student, then you might be more willing, especially in a context where a teacher is describing to you why this will be good for you, why this is a benefit for all of us. And then for teachers who have not encountered the TILT framework, when students can bring in this one pager that has some studies listed at the bottom and footnotes, they can see that when the student is asking me, why should I bother? This is actually a legitimate question. This is not a troublemaker student, this is a student who actually knows that they will benefit from knowing a little bit more in advance about this assignment that they’re planning to do. So we try to make it as easy as possible to implement. And then we also try to say only a little bit of this will make a statistically significant difference for students’ learning, so that you only have to try it a couple of times in a whole term. And you’ll probably see the kind of differences that we saw in terms of increases to students’ confidence and their sense of belonging, and their metacognitive awareness of the skills that they were practicing and developing. So if you’re doing anything new or different for the first time, yes, there’s some difficulty to that, but this one is a very, very desirable difficulty. [LAUGHTER]

John:We’ll share a link to that one-sheet document as well as to your website in general. And you do have a lot of research cited on your website. And there’s also some ongoing projects. Could you talk a little bit about those?

Mary-Ann: Yes, we are sharing all the resources that we possibly can on the TILT higher ed website, because we want for everyone to have access to this. Some of the places that benefit most are places that might have the least amount of money that is allocated for faculty development or educational development. So we want to make sure that this is accessible to anyone who would want to try it. And then the studies that we’ve done in the past, there are a few studies that have indicated to us a number of the benefits of TILT. One of the first studies we did was the national study we ran with the Association of American Colleges and Universities. It was funded by TG Philanthropy and my colleagues working on that project were Tia Brown McNair and Ashley Finley. And what we did there was we worked with a group of seven minority-serving institutions from across the country that represented every possible type of minority-serving institution, as well as a range of educational contexts like urban and rural, two-year, four-year, research university, really small in scale, large, residential and non residential campuses because we wanted for teachers to look at our results and see, “Oh, well, this worked for those faculty at that institution, and there are students like my students in that mix, so maybe this would work for my students. And in that study, we started with 35 professors at seven schools and we surveyed about 1200 students and we saw that, for the students who received the more transparent instruction, their competence and their sense of belonging and their metacognitive awareness of the skills that they were developing, those increased, those were higher for the students who got more transparent instruction than for those who got less transparent instruction. And then we also saw in that study some differences that showed us that while all the students were benefiting to a statistically significant level, underserved students were benefitting slightly more. So first-generation students in their family to attend college and ethnically underrepresented students and low-income students have slightly larger benefits than the benefits for the whole group. And then in our second study, we focused on how long does this effect last. So we worked with a group of University of Nevada – Las Vegas students. At the time we were working with that study, University of Nevada – Las Vegas had the most diverse undergraduate student population in the nation, according to US News and World Report. And we know from other studies, like Walton and Cohen’s, 2011, Science Magazine article, for example, we know that when students’ confidence increases, when their sense of belonging increases, they tend to persist longer in a course. So in courses that have higher levels of confidence and belonging, fewer of the students would drop the course, for example, more of them more likely to complete the course. And we wanted to see how long does that last. Is it just that course? And some studies indicate that this could last for a year. And what we did was we kept looking at the retention rates of these students to see how many of them were still registered a semester later, a year later, two years later. And we saw that by the time students were in their third year of university as undergraduates, those students who had received transparent instruction in one of their large gateway intro courses in their first year, those students were a little bit more likely to be still registered as students in their third year. And we’re now tracking that out to six-year graduation rates. So we saw that not only does transparency have a beneficial effect, it’s statistically significant, but that effect lasts for a good long time. And then in the state of Washington, we’re now writing up that study I mentioned with the Community and Technical College System. And I think that TILT is particularly helpful in that environment, because the population of community colleges and technical colleges is a little bit more diverse. And we have more students who belong to that underserved category of students, first-generation, low income, ethnically underrepresented. And what we’re finding from that study is we’re understanding a little more about how does transparency work, and I want to thank all of the researchers who are contributing to all of these studies too, because I’m not an educational statistician, so Daniel Richard, and Carolyn Weisz and Kathryn Oleson are contributing to this study and doing a lot of the analysis, along with help from some graduate students who have been working on this project over the years. What they’re discovering is that transparent instruction has a direct impact on students’ awareness of the skills that they’re learning, and it has a direct impact, similarly beneficial, on students’ sense of belonging. And then separately, sense of belonging has a direct impact on students’ metacognitive awareness and skills that they’re developing. So TILT has this direct effect. And then there’s this other effect between belonging and skill development as well. So we’re finding out more about precisely how TILT works for the benefit of students in these studies. And I think in terms of next studies, I want to be asking questions that really matter to populations of faculty and students around the country. So we open up the TILT research team to anybody who’s curious about this, and a number of faculty have asked about, can we say something more about how this works in an online setting, in an online synchronous setting in an online asynchronous setting, and we’ve got a few publications up on the website about that, but others are looking at that a bit more. And then we have another person who’s looking into just the impact on low-income students to see if we can find out more there about the details of how this works. And I’m really curious to see if we can work with large state systems, what can we find about the most time efficient, most beneficial ways to apply transparency and learning and teaching in community college settings. And I’ve also noticed that as I begin to do more work internationally, because I now have more flexible time to be able to do that, the colleges of applied sciences, like in the European Union, for example, they have a kind of three-year degree that is similarly focused on students’ learning something from their degree like they do here in a community or technical college that will lead them on a path into sustainable long-term employment and a career. So I think that this is going to be a really beneficial place to focus TILT efforts and to do some more research about how can we long term have an impact on not just students’ education, but how that is a pathway into a career. And I’m hopeful that we can find out more about that, like the longer long-term effect of TILT. But I’m also really open to inviting anyone who wants to do more research with the mountains of data that we’re sitting on, to discover something that is of interest to them about how students are learning, and how we can help students succeed more.

Rebecca: I really love all the resources and examples and research materials, worksheets, that are on the website. They’re really handy for folks who are starting out. We always wrap up by asking: “What’s next?”

Mary-Ann: What’s next for me, and then what might be next for teachers and students too. So we’ve talked a lot in detail about how TILT works, and how we know it works, and what more we want to discover about how it works. But I want people to remember that this is really a small effort, it’s a very easy lift that has a really large benefit from the size of that lift. And so I would really encourage teachers and students, if they’re going to do anything at all, even if they have no time to adjust any assignment prompts or to adjust anything about the way that they’re teaching or learning in a classroom. If you use any one single thing, I would say use that framework that we built for the students that has the footnotes at the bottom, and it’s called the “unwritten rules” and that framework, and I think you could probably provide a link to it, that’s what I would hope people would do next, just take that framework with you to anywhere that you’re communicating with your students. And the students will tell you how to make the work more transparent for them. Ask students what they see as the purpose, the task, and the criteria. And you’ll discover very quickly, very efficiently, how you can make that work more transparent so that all students are starting to do the work with the same understanding about what’s expected and with the same set of resources that they need in order to do it. So that’s what I hope is next for teachers and students.

Rebecca: And I hear all the faculty cheering about efficiency, and quick. [LAUGHTER]

Mary-Ann: That’s good. Yeah. So that would be the most time efficient thing to do, I think is to have students teach us more about how to be more transparent. And then in terms of researchers, I’m hoping that researchers will think about what can we learn more about? Can we learn more about what motivates students? Or what forms students’ sense of belonging? Is there anything in our survey data that would shed light on any kind of work you’re doing around that? Is there anything in our survey data that would shed light on more of the research on neuroscience and how that’s impacting learning? Or is there anything in the research that we have in our survey data that might help clarify what would be most beneficial for the very most at-risk students? So if we look at federal government statistics, National Center for Education Statistics about retention rates and graduation rates of different populations of students? Can we double down and look at those students with the very lowest graduation rates? And can we find something about TILT that would be the most beneficial for that population of students? To me, that’s a really important and interesting question. And then I really do want to be finding more locations where TILT could be useful, small scale for teachers and students, large scale for state systems or national systems to be thinking about how to apply this all for the good of students success, and for the satisfaction and time efficiency for teachers work as well.

John:If you’re finding these results of long-term persistent effects from just a single intro course, imagine what would happen if all intro courses use the TILT approach. I imagine the effect would be magnified if it was adopted at a broader level and it is being adopted at many institutions at a broader level.

Mary-Ann: I absolutely agree with you that applying TILT across the largest introductory gateway required courses at any institution would be probably the most efficient way to improve retention and graduation rates. Because if you go for the largest group of students as they enter, and you reduce the number of those students who might be thinking or doubting or wondering if they should continue, and if you increase the number of students who feel confident, who are aware of the value of what they’re learning, in terms of skills and knowledge, and if you increase the number of students who persist from the first year on, then that’s where you’re going to have the best success in increasing retention and graduation rates. I agree with you. I think that’s a really strategically wise place to invest TILT effort.

Rebecca: Well thank you so much. We’re looking forward to sharing this with our listeners.

Mary-Ann: And thank you so much for the opportunity to talk with you this afternoon, I really appreciate it

John:Thank you for all the work you’re doing.

[MUSIC]

John:If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

286. Grading for Growth

Traditional grading systems provide incentives for students to focus on maximizing their grades, rather than their learning. In this episode, David Clark and Robert Talbert join us to discuss alternative grading systems that encourage students to recognize that learning from mistakes is a normal part of the learning process.

Robert is a Professor of Mathematics at Grand Valley State University and the author of Flipped Learning: A Guide for Higher Education Faculty. David is an Associate Professor of Mathematics, also at Grand Valley State University. Robert and David are co-authors of Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, which will be published this summer by Stylus Publishing.

Show Notes

  • Talbert, R. (2017). Flipped learning: A guide for higher education faculty. Stylus Publishing, LLC.
  • Talbert, Robert and David Clark (2023, forthcoming). Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, Stylus Publishing.
  • Nilson, L. B. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Grading for Growth blog
  • Robert Talbert’s other blog

Transcript

John: Traditional grading systems provide incentives for students to focus on maximizing their grades, rather than their learning. In this episode, we discuss alternative grading systems that encourage students to recognize that learning from mistakes is a normal part of the learning process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Robert Talbert and David Clark. Robert is a Professor of Mathematics at Grand Valley State University and the author of Flipped Learning: A Guide for Higher Education Faculty. David is an Associate Professor of Mathematics, also at Grand Valley State University. Robert and David are co-authors of Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education, which will be published later this year by Stylus Publishing. Welcome, Robert. And David.

David: Good to be here.

Robert: Thanks for having us.

John: Today’s teas are:… are either of you drinking tea?

David: I am.

Robert: I am drinking tea. I’m more of a coffee guy normally, but I figured for the occasion I would bust the tea out.

John: What type of tea are you drinking?

David: I have, because its afternoon, a really lovely almost white jasmine right here.

Rebecca: Nice.

David: …really delicate and anybody who cares about tea is going to be horrified that I sweetened it with honey,

Rebecca: But it is in a really nice polka dotted mug. It is a lovely mug. Thanks.

Robert: And I have cheap stuff from the grocery store, because that’s my brand. This is a Bengal Spice by Celestial Seasoning, and my wife and I are addicted to this tea. We drink probably four or five cups a day of it, though to be honest when it’s cold out, but it usually is [LAUGHTER] here in Michigan.

Rebecca: And I have Jasmine dragon pearls today.

David: Oh, nice choice. Both Jasmine.

John: And I have spring cherry green tea. It’s a very cold wintry day here as we approach spring in upstate New York. We’re recording this a bit before it’ll be released, so by the time you hear this, we should be having the beginning of spring both here and Michigan, I’m hoping.

David: We hope, yeah..

Rebecca: Your tea choice is definitely a big wish for spring. [LAUGHTER] So we’ve invited you here today to discuss Grading for Growth. Can you talk a little bit about how this book came to be?

David: I don’t remember, Robert, if we were in person, or on a Zoom call or something, but I remember you saying at some point, I got a phrase for you: academic book about alternative grading. And I said, “You know, I need a project for my sabbatical.” And then I think it just happened from there [LAUGHTER], and so…

Robert: Yeah, we were in person, David. David and I, we’re not on the same hallway, we’re kind of on opposite arms of a T-shaped hallway. And so we run into each other, sometimes literally, at the intersection of these two hallways, and that happened one day. It was pre-pandemic, I guess. And it was just like, “Hey, Dave, I have this idea, Somebody should write a book about alternative grading. Because David and I have been using various forms of alternative grading in our classes for a while. And he was like, “Yeah,” and that was it. [LAUGHTER] But I think we were also tapping into some stuff that we have been hearing and around our Math Department and elsewhere, our colleagues outside of Grand Valley that started growing even three years ago, just a growing interest or dissatisfaction with the way grading is working. And we’ve been trying some stuff and thought, don’t you know, the best way to make change is to just get your ideas out there. And so this seems like the right place, right time for us.

John: So what types of alternative grading systems do you discuss in this book?

David: A pretty wide variety. And actually, something I’ll say, I think, both Robert and I tend to try to avoid labels too much for these, like there’s useful names to describe different approaches. But we do things like standards-based grading, specifications grading, ungrading, standards-based assessment. standards-based testing, but we look at a really wide variety of alternative grading approaches. And we’ve actually tried to come up with a framework that sort of describes what their common elements are, so that we don’t have to worry as much about names as what the useful features of them are. We found that people can really get locked into an idea of what, for example, upgrading means, and it’s not always a super useful thing to argue about with them.

Robert: Yeah, I would echo that too. Being mathematicians, Dave and I are both really into abstraction and so we look at these specific things that we see, but we’re more interested in the big, general overarching unification principles like what are all these models that are all good, and all applicable, in different places, to different levels of success, and what do they all have in common? And we do discuss ungrading, we do discuss specifications grading, standards-based grading, and a whole lot of approaches that are kind of in the in-between interstitial space, mostly through other people’s stories. I think the heart of this book and David’s real amazing contribution, what you really spend your sabbatical doing, Dave, was interviewing dozens and dozens and dozens of actual real life frontline professors, nobody is in one camp exclusively, everybody’s using some kind of combination, some kind of mix of all these different ideas. And so I think, to me, what the real contribution that our book makes is showing how different things can look, you can start from these basic building blocks, but real people with all kinds of different classes and life situations and professional situations are making this work by listening to their students and adapting appropriately.

David: I can’t emphasize enough how much variety there is in the people we’ve interviewed and the disciplines that they’re in, the types of classes they’re teaching, like it was an amazing thing to talk with all these people and see, okay, they’re able to use different types of alternative grading but absolutely across the board: in labs, in huge classes and tiny classes, and upper-level classes, and absolute freshman-level intro classes. And so it’s been just fantastic to hear how everyone’s doing this and try to put it together into something that could work for anyone, that everyone will find something useful.

Robert: Yeah, no two of them are alike either. And what’s even more amazing to me is that we had to cut a lot of those case studies. And we almost have enough material for a second book just for the case studies. We’re going to keep it on the blog, I think, unless Stylus wants us to do it. But, I mean, there was a lot out there that we don’t talk about in the book. So there’s more where that stuff comes from.

Rebecca: You mentioned that these alternative methods have some common themes or common threads. And maybe it would be helpful to talk about those common threads in relationship to the problems with traditional grading that lead people to these other alternative methods. Can you talk a little bit about that?

Robert: Well, I think one of the places that you begin to see some of the shortcomings of traditional grading is just looking at the history of where traditional grading came from. And it’s sort of a weird and checkered, and very short, history. Many of your listeners might be surprised to know that the current system of points based A-B-C-D-E-F, 4.0 GPA type of grading system is only about 100 years old. It’s a relatively recent innovation in higher education. Higher education, formally, is about 1000 years old. 1088 was the first university and the first readily identifiable grading system that we know of now is like 1890s. So it showed up very, very late to the party. And immediately it was co-opted by Industrial Revolution era approaches to teaching and learning which treated students more or less as subjects. The word “grading” really comes from agriculture, when it comes to like grading Grade A beef and grading grain and flour and that kind of thing. And pretty soon it became entrenched. And we have a situation now where when you look at where grading has led us, it’s highly questionable whether grades really do what we want them to do. And it seems almost certain that they do some things that we don’t want them to do. They lead to issues with ranking and sorting students, pitting students against each other. It’s not clear to me that the statistical validity of points-based grading is even well established. I mean, we put points on things, but they’re not really numerical data in any sense. Computing an average of points across a system of exams does not necessarily tell you how much a student knows. And that, to me, in my view, is the fundamental issue that I have with traditional grading, why I moved away from it. I just didn’t feel like the data was telling me anything.

David: I’d like to jump in on that one, in particular. A thought experiment that I think is really helpful for anybody to do is, let’s say you give a big exam, and a student takes it and they get 60%. So, most systems, that’s basically failing. What does that 60% tell you? What did they know? What don’t they know? Why did they do that poorly? Did they actually do poorly? Do they know 60% of the things that you’ve covered on that exam really well, and the other 40% not at all? Do they have a mediocre level of understanding of everything? Maybe they understand absolutely everything great, and they had a terrible day, and they had to get the kids to Grandma’s house or their work had to keep them late. Maybe they’re sick. All those things are wrapped up together, in that 60% and you just don’t know what it means. And then to add to that, if that student learns, and they really show great effort, and they can tell you later on, “Hey, I’ve got all this stuff, and let me show you how, maybe on like a cumulative final, that 60% is still averaged in and it’s going to permanently weigh them down and their final grade won’t represent what they actually came to know.

Robert: Right. And so grading really cuts against the very process that humans engage in when they learn anything. When I’m learning how to play a song on my bass guitar, and I play it once and I do terribly at it, if I eventually learn how to play it, people should not be looking back at the first time I tried it, [LAUGHTER] they should be looking at the last time I tried or the maybe the best time I tried it. That’s not how recording artists record. It’s not how athletes are ranked, and so forth. And it’s all because of this sort of Industrial Revolution era routes where we have everything measured and sorted out and put together and it inhibits growth. It sort of poisons our relationship with students, it leads to all these extrinsic goals that students now have. Rather than focusing on learning and growth, students get the idea that it’s wrong to fail at things the first time, whereas it’s actually normal to fail at things the first time and then grow from it. And I guess that’s where the name of our book came from, I forget where that phrase “Grading for Growth” popped up. It was way before we were asked for a title for our book. But it’s like that’s what we really want. We want to have a system, even if we must call it grading, we want it to be a system evaluating student work that focuses on and encourages growth as you are learning because that’s one of the great things about being human is that we do grow and we do learn from our mistakes. So where is it in our assessment systems? That’s kind of the fundamental question we’re asking.

David: So it might be helpful if we talk a little bit about this four pillars framework that we have in terms of where we go with that. We kind of spent a while trashing traditional grading here, and I could do that all day. So what did you do instead? What are the things you want to do instead of these things you don’t? And so from my point of view, the most important thing I want to know out of a grade is what does the student actually know? What do they learn? And so, as I said, we have this four pillars model, we talked about four pillars that any good rating system should have and one of those is really a clearly defined description of what it is you’re assessing. So we call it clearly defined standards. I don’t want to catch on to the name standards too much there, but a description of what it is you’re assessing and what it is that matters in that assessment. So how do you know that the student has achieved it? And what is it that the student has achieved? So in that way, that takes care of this issue of what does that 60% mean? 60% of what? And if you’re grading based on a specific thing that you’re testing a student about, then you can say, okay, maybe they’re achieved that or they have not achieved that specific goal. And then we try to incorporate in our other pillars, this idea of feedback loops. So humans work on feedback, they work on trying things multiple times, they work on trying and failing, and having to come back again, and that it should be the ultimate level of what they know, that really matters. So focusing on feedback, rather than focusing on numerical rates. So focusing heavily on feedback, on what’s happened, and how that relates to the standards or the specifications, or whatever the description is that they were trying to reach, and then making that feedback actionable so the student has a chance to actually act on that, and either through a new attempt show that they’ve got that idea, or through revising previous work, show that they come to understand it in a way that counts fully, so that they’re not penalized for needing multiple attempts, just like Robert was talking about with his bass, we’re not going to look at well, the first time you tried, it didn’t work out, too bad. We want to know, ultimately, where were they, so we shouldn’t penalize multiple attempts at understanding something. And finally, it feels like a technical thing, but it’s actually a really big move, moving away from points or percentages and instead, if you’re gonna put a grade on work at all, to make that grade something that’s sort of a descriptor of the feedback, something that basically says, you’ve met this standard or you’ve met these requirements or you haven’t and gives the students an idea of where to go next: you need to revise this, you should try this again, you’ve made it, something like that.

John: I know you’ve experimented with a variety of alternative grading systems. Could you talk a little bit about how you view some alternative grading systems in your own classes?

Robert: Sure, I’ll jump in on that. So around 2017, I described this in the book in my origin story where I came from with this. I had just taught a calculus class and I had just a terrible experience with grades. One student in particular, a fantastic student, she was the epitome of what we’re discussing here. She was on a 10 day delay from the rest of the class and after those 10 days were up, she’d be at the top of the class, but the exam is today, and so her exam scores just went down, down, down, down, down. It was just an awful way for a student to experience my subject. And so I started casting about and I read this book that had just come out by Linda Nilson, who is a legend in the area of professional development, and my former boss at Vanderbilt University. And it’s called specifications grading, and I said “What is specifications grading?” So I read this book, and I was converted on the spot. And I’ve been using specifications grading in almost all of my classes, except for one instance of ungrading last year, ever since. And so, I’m in various forms, and always tweaking, always experimenting, and trying to change things up. But, specifications grading is exactly along the lines of these four pillars that David is mentioning. There are clear content standards that we call specifications. The idea is not to give like points, but marks, to use a sort of a non-American term for those, marks that indicate progress, like you’ve met the standard, that’s what you get instead of a 10. Or you’re progressing, that’s what you get instead of a five, and giving lots of helpful feedback and especially letting students retry things without penalizing them for retry. To me, that’s like the thing that drives the loops. That’s been my main sort of area ever since for the last several years,

David: I’ve used a pretty wide variety of different alternatives. It’s come to the point for me where I think about all these different types of alternative grading systems that have names like standards-based grading, or specifications, or ungrading as elements to put in and they may be appropriate in some classes and not in others, depending on a whole bunch of factors. For example, I’ve taught intro level classes where I use a lot of standards-based grading where that means I’ve sort of divided up the topics into fairly fine-grained standards describing what a specific skill is, and what it would look like for a student to achieve it. And then the grade is essentially based on have they shown me that they thoroughly understand that, they can do the thing described in each of these standards and grades are based on meeting or not meeting those standards. And that can look like pretty traditional assessments, quizzes or exams or homework or such things. But instead of getting an overall grade, they get a mark, like Robert was talking about, for each standard. Yup, you’ve achieved this. Nope, you haven’t yet. Maybe they need to do it a few times. But that’s one element. It’s really useful for sort of discrete skills, intro level things. I’ve used specifications in a similar way that Robert is talking about. One of the strengths of specifications is that you sort of look at a student’s work holistically and say, “Have they overall shown me they can put these ideas together, understand the concepts, use all the different things that I think matter within this assignment,” and that works really well on written work… We’re mathematicians where students write proofs, it’s a detailed explanations of why something’s true, or in a project or in a portfolio or something longer than that. It’s really useful, maybe in upper-level classes or in places where I want students to show synthesis. And I’ve gotten pretty deeply into ungrading, or at least as I use the word ungrading, meaning removing grades entirely, even marks, in the way we’ve been talking about as another element that can be useful either in an entire class or just as individual things to do within some assignment. So I’ve covered a lot of this, I continue to push on what’s most appropriate in different classroom situations and different student situations.

John: I’ve tried some of the same things too. But one concern from a lot of faculty who have tried mastery grading systems or mastery quizzing and those types of things, is that it can put a lot more work on the instructor to do the grading on multiple attempts. In Linda Nilsen book, she suggests giving students a certain number of attempts, but limiting those. How do you keep the work manageable so that you’re not spending all of your time grading additional attempts, as students are working towards mastery.

David: Yeah, the thing you mentioned from Linda Nilson’s book, attempts without penalty doesn’t mean attempts without limits, and that is really important. And yeah, limits like that are one way to do it. There can be limits on number of attempts, or on frequency of re-attempting something or revising something once per week, once total, something like that. Those sorts of limits can just be really useful. And I know, I always encourage new instructors or people just starting to use these systems to start out limited, and then add flexibility because that way, they will be able to do it if they need to, and not do it if they don’t want to.

Robert: Yeah, you know, another thing too is, not just in the reassessments, but in your overall design of your course, including the grading system, you got to keep things as simple as humanly possible. This is something David and I just hammer home constantly, like every other paragraph ends with keep it simple, because people who want to mess around with grading systems are people who like tinkering with systems, and I am one of those people, and I know that when you start tinkering, it’s very hard to stop. And you end up with this massive Rube Goldberg like device that is your class and to you, it looks beautiful, but when you face it towards students… I mean, the first time I did specs breeding, I had 68 learning objectives that students had to meet throughout the semester and it was a nightmare. At the first of the semester, I thought, I’m a genius. I mean, look at this beautiful grading system I’ve devised. It’s going to revolutionize everything. And the only thing it did was give me a grading jail for four months. Just keep it as simple as humanly possible, and you’ve got to cut the work off at the source, I think. David and I wrote a book about grading, but we don’t like grading. I mean, nobody really likes the process of it. But it can be made at least a fulfilling undertaking if you are orienting it towards growth. But you still don’t want to be doing it 8-10 hours a day. So keep things as simple as humanly possible and put some limits on that. It’s okay to create a little bit of scarcity when it comes to the reattempt side of things. The feedback needs to be helpful that you give to students, but it doesn’t have to be incredibly lengthy. It might just be a couple of sentences, it might just be: “You did this really well, and this needs work.” …to kind of keep it right to the point, and then that helps the students who are on the other end of this too also experiencing a workflow possibly overload. And I worry more about them than I worry about myself. And so this makes it more likely they’re going to read your feedback if it’s shorter, more concise, and it makes it shorter to give.

David: Yeah, I want to emphasize that it can be a lot of work for students as well. Something that I suggest a lot is, if you want to use a system that emphasizes these pillars, especially reattempts without penalty, you’ve got to then think about those re-attempts as part of the regular workload of your class. Really what it means is you’ve got to think about the process of learning, that whole process, is part of the time that students are going to have to invest into it, and that you need to value that in your timeline for the class. So I like to think of maybe if you expect a student to work, however many hours per week on your class, account for whatever time each assessment takes, add another half for how much they might need to do revising or reassessing or reworking on things. And that can lead to really difficult decisions. You’ve got to cut stuff out if that’s going to happen. I don’t know, basically, anybody who looks at a classroom teaching and says, you know, there’s not enough in here. And so you got to make those difficult decisions, but it is worth it. Because what comes out of it is a student knowing things that are still in there way better than they would have otherwise.

Rebecca: I think one other barrier that faculty sometimes face is learning management systems, and how they guide you towards particular kinds of grading systems and evaluation systems. I wonder if you can share some tips that you have from your own experience of dealing with a learning management system to help you and your process rather than getting in the way of implementing some of these techniques.

Robert: We’re a Blackboard campus, and we’re making a transition right now and I’m honestly not sure how the new system handles those. But in at least the next to most recent version of Blackboard there was a thing called a schema, which is basically a way of going into your gradebook and defining a way to map number inputs to text outputs. And so I can set an assignment up to be graded on zero or one point. And then I can only enter in zero or one. But I could tell Blackboard if I enter in a one put “meets expectations” in the gradebook, and if I’ve put a zero put “does not meet expectations” or a happy face emoji and a frowny face emoji or whatever I feel like doing. And I have zero experience with any other learning management systems over the last 15 years, so I can’t really speak to that. But I know that many memory management systems allow you to customize the way that your numerical inputs present themselves in the students’ gradebook. And if you can find out how to do that, that’s the way to go, as far as I’m concerned.

David: And even if you can’t, the thing Robert mentioned, that essentially everything is a zero or a one, these systems all want to use points. So just refusing to play that game by making everything a zero or one. Even if students are seeing the zero or the one, it’s a lot easier to interpret the zero or one as success or not, than it is if you have points showing up somewhere else, and you’re trying to convince students No, no, no, when I put in a five, that meant something that doesn’t mean five points, that’s not going to fly. So limiting it to just a zero or one, a complete or incomplete, something like that works pretty well. That’s also true for spreadsheets. So if you just want to keep your info in a spreadsheet, they still want to work with numbers for the most part, but you can think of everything as a zero or a one. And then you start to change your mind from averaging or totaling things to counting. So I’m counting how many standards a student has completed, I’m counting how many assignments they’ve met the specifications on. And that’s really the same as saying, how many ones do I see here? One last thing is, everybody seems to have a different LMS. Even those of us who use the same one don’t quite use the same version of it [LAUGHTER]. So the best thing to do is to find someone who knows it really well and talk with them and say, “How do I make this type of thing display?” …and they probably know a way,

Rebecca: I’m gonna have this conversation with John about our LMS.

John: We’re using Brightspace from Desire to Learn, and it does have those capabilities.

Robert: And one thing I would say is that these days, as the ideas about alternative grading get more and more airplay, I think the tech companies are starting to listen to these things. I mean, you’re actually starting to build these things in as a competitive device like “Ah, you should adopt our LMS, because we can let you do ungrading and you don’t have to deal with all the hassle of hacking your own LMS. So I feel like ed tech companies should be paying attention to all this great interest that’s swelling up, it’s like this is a way to earn customers, honestly.

David: Something I’lll say most LMS’s do seem to do pretty well is to allow you to get feedback in a variety of ways. And so if you can de-emphasize that grade portion, just 0-1 or hide it or something, especially like I accept all assignments through Blackboard nowadays, giving feedback’s pretty easy on that. So that, in my mind, is a benefit.

Robert: Yeah, Blackboard does a really well, actually, that if you submit a PDF into Blackboard as an assignment, it’s easy to leave comments on it, and it keeps all the versions of it. So you can go back and easily see the students trajectory from the very first draft that they do all the way to the present day. And I think that’s really powerful, actually.

Rebecca: I’ve used rubrics to set up essentially specifications as well, like it’s met or not met.

David: Absolutely, yeah. If you know what you’re doing setting up rubrics, I don’t, that’s the problem for me. But if you know what you’re doing, that’s fantastic and it can also save you time.

John: Since the pandemic, there’s been an explosion of interest in ungrading. Does this approach work as well in all contexts, and for all students?

Robert: People are starting to use ungrading to mean anything other than traditional grading. And I think that’s leading to a lot of confusion, honestly. I was working with some teachers recently who were saying that, “Oh, I’m using ungrading but don’t have time, I’m being overwhelmed by all these quests for reattempts.” And it turns out, they weren’t using ungrading at all, they were totally grading thing. When we say ungrading, we mean like a specific approach to evaluating student work where nothing gets a mark, literally ungrading. You get a letter grade at the end of the semester, because most places require that, but it’s all based on a term that’s come up recently, that I really prefer, is collaborative grading. You’re working together with the students throughout the semester to talk about the quality of work, but nothing is getting a mark on it. I have a comment for you and you can either choose to iterate on that comment with a feedback loop or leave it alone. At the end of the semester, you’re going to sit down together and collaboratively determine what your course grade is based on the body of work that you’ve accumulated in some sort of portfolio situation with some clear standards for what an A would look like, a B would look like, and so forth. So that’s what we mean by ungrading. The explosion of interest in “ungrading” may or may not be all about ungrading the way we’re referring to it. So I just want to be clear about the terms before we answer this question.

David: And just add even another thing, it’s both that sort of umbrella term and what Robert just said and the general philosophy and a buzzword that people attempt to apply to random things they use. It’s unclear what someone means when they say ungrading.

John: When we’ve been talking about it, we’ve been talking about it exactly as you described as an extreme form of alternative grading where there are no grades other than one that is decided in consultation with the student at the end of the semester. And sometimes at midterm if that’s required in the institution. From what we’ve seen, this is used quite a bit in humanities fields, but it tends not to be used very much in the STEM fields. What are some of the barriers or some of the concerns that might be raised for ungrading in terms of in what contexts it may not work as well.

Robert: I have tried to fully ungrade a course once, and I might use it again. But the issue that I was coming up with with my students, is what I say is that ungradiing works precisely as well as students’ ability to self assess. If you have students who are struggling with the ability to self assess, or they’ve never tried it, or they’re new students, or younger students who maybe are still emerging with that idea… the whole thing about ungraded is that it’s predicated on students looking at their own work and self evaluating. And if that’s an issue, then I’m not totally sure that ungrading, as we’re defining it, best serves that population. I know I struggled with it. And this can be more me than anything else. Maybe I just don’t know how to draw students out and I need to work on that. It’s an area of growth for me, certainly. But if you have a population of students, for whatever reason, whose ability to self regulate or self evaluate is questionable, or kind of low, we’re not going to say bad students, because we’re all growing… it’s the whole point of our book… if you deploy ungrading with that population, you might be sort of targeting the weakest point in your class. And so maybe, in that case, some marks wouldn’t be such a bad thing, like use a specifications grading approach where the marks are just like: you’re progressing or you’ve met the standard. Just some kind of simple mile marker that shows students where they are. When I used this, once, it was an upper-level class, it did have a lot of writing in it, a math class. I would leave comments on their paper and I would just get questions like, so does it meet the standard or not. And I didn’t like to sort of be telling students this, but I felt like this is the way I helped them. I said like, “hasn’t met our standard yet, here is what you need to do.” So I thought, if I’m going to do that, why don’t I just put a mark on it? It seems like I’m just beating around the bush and trying to be cool by not giving marks and it’s like I’m thinking more about myself than I am about the students at that point.

David: I think that a general thing that everybody needs to think about when using any kind of alternative grading is, this is something new for students… almost guaranteed. And like any other new thing, it needs time to learn, you need to encounter it a few ways, you need to try to make sense of and probably fail initially, and then come back and you’ve got to work to understand it. Ungrading is sort of an extreme form of that. There’s more that needs to be understood about how it works, there’s more skills that need to be built. And I think, as instructors, we need to think about that and think about teaching how our evaluation systems are working and how the grading is going to work. And think about what skills we need students to build up in order to successfully engage with those. And ungrading asks a large amount of that. And I say this, as I very much enjoy using ungrading also in some upper-level math classes and I think it’s worked pretty successfully for me. But this sounds funny to say about something called ungrading, it requires scaffolding, it requires helping students build up that ability to self evaluate, advocate for themselves, to be able to understand what it is that matters in that particular system. But what I just said is also equally true for standards-based grading or for specifications grading, but with different skills that they’re building up. It’s also true for traditional grading, although it feels more familiar to students, but it still requires some scaffolding for them to understand what exactly is going on in the evaluation in this class.

Rebecca: Can you talk a little bit about some of your own experiences with ungrading and how you have done some of that scaffolding, what some of those early stages of scaffolding might look like, to understand the system.

David: So initially, I was having I think it was three check in meetings per semester, where I would meet with every single student in my class, like a 10- or 15-minute discussion, and we would talk about where they’re at. And that inevitably brings out students’ questions, and they don’t understand this part of they don’t know what that is about. And so that was one thing that I did, right? Those were the collaborative aspects of ungrading, and it was overwhelming, it was too much to meet with every single student that often. I still love being able to meet with students. And I use that as one of the tools in the toolbox. But nowadays, to sort of help build that up, I have a system where… this is actually done through homeworks for me… it’s like periodic homeworks and there’s some mathematical content. And there’s also some little additional thing where I ask a question, and early on, it might be something like, “Okay, what is a specific goal related to… and then I might have some of the math related content in the class or as a specific goal you’d like to work on related to this.” And that’s a fairly small thing. I ask students to say something along the lines of “I really want to work on improving my understanding of this one thing” and I can either respond to that as “Okay, great. Here’s the way I can suggest that you work on that,” or I can respond to it as… well, I don’t say it this way, but… “I can see that you’re having trouble elucidating the goal and so let’s talk about that” instead. And then another week or two down the line, there’s another maybe slightly higher-level task having to do with assessing themselves or thinking about their progress in the class. And so they’re scaffolded and I can, at each stage, see if a student is succeeding, in sort of the way that I’m looking for them to be thinking about their progress. And if not, then I can pull in a, “Hey, let’s actually meet and talk about this. And then I’m gonna give you a task to practice with that.” It’s really individualized, which can be a tough thing. But for me, it’s really fun to actually see students grow in that particular way, in addition to growing in the mathematical content.

Rebecca: So faculty that want to move to less traditional methods, what are some small steps a faculty could take to get started?

Robert: Well, I really appreciate that question. Because we don’t ever want to give the impression that you have to go all in on one particular grading system in order to like, be cool, or be a good professor or whatever, because everybody’s in a different position. Some people may not be in a position to ungrade, and we are never going to come out and say, “If you really care about students, you will do ungrading or you’ll do standards-based grading or whatever.” It’s like people care about students all over the place and just can’t or have mitigating circumstances. So I would say we laid out these four pillars: clear content standards, helpful feedback, marks that indicate progress, and reattempts without penalty. If a professor takes one of those pillars, and kind of drills into it on just a handful of assignments, that’s definitely progress in the right direction. So for example, you could take your next test you’re gonna give, experiment with letting students reattempt it, or reattempt different versions of it. So go one test, one class, allow reattempts without penalty, or give feedback that really gets to the point, not gives points, but gets to the point of what’s being done well and what isn’t being done well. Just pick one of these four pillars and just go with it. There are some small scale methods you might think about, let’s say, instead of grading your final exam, how about ungrading your final exam. Just say you need to lay out some standards for what constitutes successful work on your final exam and maybe that’s like 70% on the content. And then you got to write some reflective essays, and say, if you don’t meet expectations on the final exam, that’s a minus on your course grade, or something like that. So you can roll in aspects of alternative grading systems, whatever seems to resonate with you on small-scale individual assignments. And that’s really good progress. It’s really good data for you as the instructor too. You run these experiments, you ask students how it went, and then you just make notes and you iterate on that. It’s exactly the same thing that we asked students to do in these grading systems. You try something, you get feedback on it from a trusted third party, you iterate on it and try again.

David: I want to add one thing, because this is something that I always see happen when people first start using some kinds of alternative grading. Just for yourself, write a really quick list of what you think the major topics are in your class, aim for 10 to 20 things that are the important things to get out of the class, and then go pick some of your exams or quizzes, or whatever, and match them up. “Oh, on this one I was addressing these things and on this one, I was addressing those things.” And that can help you actually think about, “Oh, you know, I’m not really addressing this thing in my assessments at all, that I said was really important.” Or “Holy Moly, I have hit this one thing over and over and over all the assessments at the cost of not covering a bunch of others.” The first time you do this, even if you’re not showing students what the standards are, it can be amazing. “Wow, I’m not assessing what I say matters.” And that can adjust just how your assessments are focused.

John: Sounds like you’re advocating a backwards-design process where you start out with clearly defined objectives, and then you make sure that there’s alignment. And also, I think it might be important to make sure that students see the connection between those things, and that it’s a transparent process, which I know is something you’ve advocated in terms of your work on assessment.

David: Absolutely. And sharing your reasoning with students I think is very valuable, bringing something in from above and saying we shall now do it this way, because I said so, doesn’t work and you’re gonna get a lot of pushback, but talking with students about why you’re doing things and how it’s good for them, that makes a huge difference.

Robert: And don’t call it an experiment, okay. [LAUGHTER]

David: Yeah.

Robert: I’ve read some syllabi lately. It says, I’m going to be experimenting with this experimental new experimental grading system. And of course, you know what that is going to sound like to students… that I’m the subject of an experiment. What am I doing here? And so you’ve instantly killed your buy-in.

David: I’ll actually say, I don’t even give names to what I’m doing. So I am ungrading a class this semester, I have never used the word ungrading or any other name for it. I find that giving a name to things like that can help people sort of reduce it down to like a one dimensional idea, rather than engaging with what it actually is. I’m just “Oh, this is what we’re doing, and here’s why we’re doing it this way.”

Robert: Yeah, I have to share this story. When I started doing specs grading I did actually put we are using a thing called specifications grading on my syllabus. So, I got a call from our research office on our campus, they had gotten a call from the parent of one of my students complaining to them that I was experimenting on their child without informed consent, that I needed to give an informed consent form to my students before this happened. And I thought, well, that’s a weird flex from a parent and they call up the Office of Research Compliance [LAUGHTER] and complain about the Professor, but as you can just see even just the barest mention… Just tell students what you’re doing, and why you’re doing it. If I have to name it, nobody cares what the name is, it may not even have a name. And all this we mentioned before, that most of the people we interviewed for the book are not using any sort of canonical form of anything. It’s just like a little bit of this, a little bit of that, like cooking in your kitchen, you don’t use just one spice, you combine them, that’s where you get your unique flavor of what you’re doing.

John: Would it be fair to say that, given the title of your book and focusing on grading for growth, that one of the things you’re addressing is how to shift students from focusing on trying to maximize their grades to maximizing their learning?

Robert: Yeah, absolutely, yeah. I think a lot of frustration that I know I feel and I hear from other faculty is students tend to spend an order of magnitude more time talking about what they have to get on exam X to get grade Y than they talk about the actual content in the course. And we teach math, and I think some of the students who struggle the most with our math are doing like these amazing statistical calculations to try to figure out what the average has to work out to be. It’s like, “where was this in my actual math lesson?” I remember riding the bus home one day, from our downtown campus back to where we live, and I overheard this conversation and I said “Oh, this is this has got to come to an end.” [LAUGHTER] And absolutely, we want students to focus on their growth and I think students want to focus on their growth, too. I mean, students really, in the end, don’t want to think about all this stuff. They want to think about, “How am I growing as a human being? Is when I’m doing really meaningful? Where am I as a learner? Do I have any value in this vast educational system that I’m entering into?” And so if we can even just orient one small thing, like our grading system, towards convincing students that they have value, and are human beings in a stage of development, I think that’s worth it.

David: And that means we need to believe it, too. So I like to tell people who are thinking of using an alternative grading system, think about why you’re doing it, and what your students are going to get out of it. But definitely approach it as like, “Okay, I care about my students, and I want them to succeed.” And we’ll sometimes talk about incentives… grading systems set incentives in a class, and that can feel sort of like you’re trying to mess around with people and then incentivize you to do things a certain way. Traditional grading systems set incentives that are really kind of perverse towards learning. And if we can change those incentives, or just remove some of the ones that aren’t as good, that’s really a lot of what we’re aiming for here. And yeah, just respecting how people naturally want to learn anyway, that just allows them to do what they are as human beings going to do anyhow, or what they would naturally do.

Rebecca: So everyone wants to know, when can we have your book in our hands?

Robert: Well, it looks like on the Stylus website, it says July. So that’s what we’re going with for now. [LAUGHTER] I guess it’s sort of out of our hands at this point. We have one more round of copy edits to do. But it says July, you can preorder it now through the Stylus website and should be in your hands this summer.

John: My preorder has already been sent in. [LAUGHTER] So I’m waiting for a copy.

David: Awesome.

Robert: Awesome. Thank you.

Rebecca: And you mentioned earlier something about a blog. Do you want to tell us a little bit about that?

David: Yeah, if you’re interested in some of our ideas, go check out our blog, it’s gratingforgrowth.com …all spelled out. So, Robert and I started this sort of as a way to workshop some of our ideas for the book. So you can see proto-ideas. And we’ve also posted some examples of some of the case studies of some of these interviews that I have done with people using a lot of different alternative grading systems there. Every week, we post something new, or we have guest posters. There’s a really huge variety of ideas that show up on there.

Robert: And if any listener wants to contribute a guest post, we are accepting applications at this time… not really applications, just reach out. And we’re looking to hear. Now that the book is kind of close to coming out, we’ve had this discussion, you and I, David about where does the blog go from here once the book is out, and I feel like a great use of our time is to elevate the voices of other people. I mean, there are so many people out there doing great work, who are not education rockstars, they’re not on the lecture circuit or whatever. They’re just rank and file heads down people doing great work in the classroom. I love those people getting their ideas out there and seeing what they’re doing. So to me, that’s been a really rewarding experience.

David: So gradingforgrowth.com.

Robert: Right. It’s actually a substack. You can subscribe to it. It is free. It’s never going to cost anything and it comes every Monday. Except for taking planned breaks for holidays, we haven’t missed a Monday in two years.

David: Well, now you’ve cursed us, Robert. [LAUGHTER]

Robert: No, I put the pressure on me because then I got to step up. I think I’m next, actually. No, you’re next.

David: Oh, crap. [LAUGHTER]

Robert: Sorry, Dave.

John: We will put a link to that in our show notes.

Robert: Great.

John: Are there any other topics that you’d like to emphasize?

Robert: Well, I think I would just re emphasize simplicity. I mean, simplicity is the ultimate sophistication. I think Leonardo da Vinci said that. And so, when you’re designing a class simplify as much as humanly possible, and no more. But I think your students are going to be well served by less, honestly. I think we tried to do way too much in our classes, and we will be well served by this too. I mean, I have another blog that deals with productivity and time management in academia. And that’s the key thing. I mean, a lot of faculty struggle with overwork and burnout or just doing too much and there’s a lot of stuff you can say no to and I think keep things very, very simple and that can be a whole lot. podcast about how to simplify your life in higher education, especially applyied to course design and grading systems. You don’t want to make choices in July when you’re planning your fall courses that you end up regretting by October.

David: I will say that something that did come up earlier, briefly, but there’s not one right way to do it. You should not feel like you must go all out ungrading, you shouldn’t feel like you have to do specifications grading exactly the right way. Find something that works for you, look for models, pick and choose the things. If you look at something and say, “How could that possibly work?” then don’t do that. And yeah, we’re not judging you. Find something that works well and it’s going to be good beer students.

Robert: Yeah, and especially the last thing you just said, David is so important. All this stuff is predicated on actually communicating with your students. Don’t just sort of wait for the course evaluations. You have to really get in and talk, actually talk with your students on a regular basis and get feedback and see how things are going. And that’s one of the great strengths of all these four pillars sort of oriented approaches is that it really does get students talking with each other and with their faculty members. And I think that’s, maybe in the end, the best thing about all this.

Rebecca: Well, we always wrap up by asking what’s next?

Robert: Well, for me, I know I’ve been spending the last two years not only as a faculty member, but working in our president’s office under what’s called a presidential fellowship. And I’ve been coordinating large-scale institutional cross-institutional teaching and learning initiatives, mostly focused on active learning spaces. And so that’s been a particular focus of mine for a few years. That is wrapping up now. But we’re looking at maybe some interesting and exciting extensions of that idea where we’re going to try to surface and looking for what our faculty are already doing in terms of instructional innovation. There’s a lot of stuff out there, like we said earlier, but even in our own institution, a lot of faculty are doing really innovative and creative things and effective things but are kind of working under the radar. So the President and I, and those around us, are kind of thinking how can we look and find these faculty, get them together, elevate what they’re doing, and sort of make Instructional Innovation a normative practice at our university. So that’s that’s kind of what’s next for me is figuring out how to make that work on a practical level.

John: Sounds like a wonderful plan. I know, we’ve been talking about some ways of doing the same thing at SUNY-Oswego. And it’s a challenge.

Rebecca: It’s no small task. [LAUGHTER]

Robert: It is, it is.

David: Something that I am getting more and more interested in is what helps instructors be successful in any kind of educational innovation, not just alternative grading and changes in their pedagogy, anything like that. But especially when it comes to grading, there’s so many variations. And in so many different situations where instructors are working, what are the things that help them best succeed? And what are the things they do that help their students buy in and succeed most? And so looking at those sort of things from a more fine grain perspective, I think it’s going to be somewhere that I’m going next.

Rebecca: That sounds exciting.

John: It does. And I hope you’re both come back and talk about the success of these initiatives in future podcasts.

Robert: We’d very much like to.

David: Absolutely.

Rebecca: Well, thank you so much for joining us and sharing your stories. And I know that many people are looking forward to your book.

Robert: Thanks a lot for having us.

David: Yeah, it was great to be here. Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]

284. Learning That Matters

Many graduates describe their college experience as being transformative, changing how they view the world and their role in it. In this episode, Caralyn Zehnder, Karynne Kleine, Julia Metzker, and Cynthia Alby join us to explore the role that college faculty can play in creating transformative learning experiences.

Caralyn is a senior lecturer in biology at the University of Massachusetts-Amherst, Karynne is the former Dean of the Division of Education at Young Harris College, Julia is the Director of the Washington Center for Undergraduate Education at Evergreen St College, and Cynthia is a Professor of Education at Georgia College. They are the authors of Learning that Matters: A Field Guide to Course Design for Transformative Education.

Show Notes

  • Zehnder, C., Alby, C., Kleine, K., & Metzker, J. (2021). Learning that matters: A field guide to course design for transformative education. Myers Education Press.
  • Mezirow, J. (1991). Transformative Dimensions of Adult Learning. Jossey-Bass.
  • Selingo, J. J. (2013). College (un) bound: The future of higher education and what it means for students. Houghton Mifflin Harcourt.
  • Selingo, J. The Future Learners. Pearson.
  • Learning that Matters website
  • Learning that Matters: The Course Design Institute

Transcript

John: Many graduates describe their college experience as being transformative, changing how they view the world and their role in it. In this episode, we explore the role that college faculty can play in creating transformative learning experiences.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Caralyn Zehnder, Karynne Kleine, Julia Metzker, and Cynthia Alby. Caralyn is a senior lecturer in biology at the University of Massachusetts-Amherst, Karynne is the former Dean of the Division of Education at Young Harris College, Julia is the Director of the Washington Center for Undergraduate Education at Evergreen St College, and Cynthia is a Professor of Education at Georgia College. They are the authors of Learning that Matters: A Field Guide to Course Design for Transformative Education. Welcome Caralyn, Karynne, Julia, and Cynthia.

John: Today’s teas are:… Karynne, are you drinking tea?

Karynne: I am and I was joking yesterday that I would have to go to Starbucks and get mine because all I have is Lipton, and I did. And so I’m having some Earl Grey [LAUGHTER] in my Hawaii Cup.

Rebecca: …where I would really like to be during our impending snowstorm. [LAUGHTER]

John: Julia?

JULIA: No, actually I am drinking coffee out of my trusty thermos as I do every morning. [LAUGHTER]

Rebecca: …there’s always one ,Julia.

John: Well, at least one.. And Cynthia? [LAUGHTER]

Cynthia: I am drinking a Tazo tea called glazed lemon loaf.

John: I haven’t seen that one.

Rebecca: It smells really tasty. I’ve had it. The smell though, is what really gets it.

John: And Caralyn?

Caralyn: I have a handpicked hand dried sweet fern and sassafras tea that my 10 year old who’s now into wild foraging blended for me.

John: Wonderful.

Rebecca: Well, that’s amazing. Can I have I have 10-year old? [LAUGHTER] I have some Awake tea today, despite the fact that it’s two o’clock in the afternoon.

John: And I have Darjeeling tea today.

Rebecca: That’s a different choice for you, John.

John: It is. I was looking for things I haven’t had recently. So I picked that one.

Rebecca: Score one for you, fail for Rebecca. So we’ve invited you here today to discuss Learning that Matters. Can you talk a little bit about how this book came about?

Caralyn: Well, we, many years ago, were all faculty together at Georgia College. And it started with Julia and Cynthia started a group focused on course design. And it morphed into what became the innovative course building group. It was this grassroots… sort of bottom up… we wanted more support, and collaborative work towards teaching. And we all began working together through this and doing workshops. And so we decided to write the book that we wished we had had, when we had first started teaching. We wanted it to be based in theory, but really practical, have a lot of strategies, be really conversational, and be collaborative, and really encourage people to work together. Because we found that sometimes teaching could be so isolating that working together and talking with other people was just something that gave us so much support and we enjoyed, and we wanted that for others.

John: This book is designed to help faculty create transformative learning experiences. What constitutes a transformative learning experience?

Karynne: Well, for our book, we actually used a Mezirow’s theory and then work really from John Dewey. And our definition is about fundamental change that learners undergo, if it is a transformative education, whereby they see themselves and they see the world differently. I taught teachers and I would always tell them, the person you will be when you leave this program is not the person who you are now. So it involves a lot of reflection, whereby you have an experience, you process that experience, and then you make meaning of that. And that changes how you are viewing yourself and the world.

Rebecca: So reflection is a critical part of that practice.

Karynne: Absolutely. And that’s really what we get from Dewey is the importance of that for learning

Rebecca: So, you start a chapter with a pre-flection. Could you explain to our listeners, what this is and why you use this approach? And how we could use it in our classes?

Cynthia: Yes. So it’s one or more questions that we have at the very beginning of the chapter. And I feel like they are just gold. I thought that for a long time. I’ve always enjoyed having individuals do some thinking upfront really before we dive in. But then in a recent study, I’m going to say it was probably 2021, around there, students who took a practice test, who answered questions before learning the material, outperformed their peers who studied it more traditionally, by 49% on a follow-up test. So then I thought, well, heck, I think these pre-questions are even more valuable than I ever imagined. And when you think about why, it makes a lot of sense, because first of all, some pre-questions, some pre-flection, gets people in a good headspace and it’s got them thinking along with what it is you’re about to introduce. I think it stimulates anticipation, because now that you’ve answered some questions, you’re curious to see are the authors going to agree with me? Disagree? What’s going to happen? And I think it can highlight gaps in your knowledge that if you answered some questions previously, and then as you read, you might think, “Okay, well, yes, I said that in my pre-flection. And oh, yes, I said that, oh, but I didn’t think about that piece.” I think it kind of shines a light on those pieces that maybe you hadn’t thought of before. So I just really, really highly recommend that not only does it make good sense for a book like thispre-free questions. And when I have students reading something for homework, I always have some pre-questions that I asked them to answer before they even ever start reading.

John: One argument for it too, is that it helps activate prior knowledge, it gets students starting to make connections, recalling what they already know, and sets a frame for them to put new material into that framework and elaborate on what they already do know. It’s a wonderful strategy and I should do more of it myself. [LAUGHTER] I advocate that very often. And I don’t do it as much as I should.

Rebecca: I know one of the things that I’ve discovered in using some of those strategies is that sometimes a topic is familiar [LAUGHTER] and so familiar is different than knowing. And so sometimes doing an activity like that can help someone recognize that it’s something they’ve heard of before, but they don’t actually know that much about it.

Cynthia: Absolutely.

Caralyn: And especially if it’s something where you know that there’s going to be some misconceptions or things like where the topic, how it’s described, maybe outside of your discipline is not the same as how it is inside, or the terminology has specific meanings. And it’s so good for uncovering that and so much more powerful than me standing up in front of the room, just talking about it.

John: One of the anchor concepts in your book is the principle of teaching towards equity. What are some of the ways in which faculty can work towards creating a more equitable classroom environment?

JULIA: One thing I want to start off by saying is that sometimes, because the stakes around equity are so high, we get a little overwhelmed. And when we start to think about how do I teach towards equity, and one of my lifelong goals as a faculty developer is to demystify the concept of teaching for equity. And so I like to say, at its core, it’s the process of humanizing the learning environment. And so what I mean by that is just approaching each and every student as a unique human with their own story and understanding that the story that they bring into the classroom will impact and influence how they learn, when they learn, what they learn. And then I also want to say to these folks, because I was there, and this idea, like how am I going to make the world more equitable? It’s such a big job. But really, we all as human beings have the innate tools to do this, because we’re social beings that live in a social environment. And we have a lot of practice in all kinds of parts of our lives, learning how to create relationships, how to build communities, how to live in relationship with each other. But it can be challenging in teaching, because we’re working against some pretty powerful social forces that lead us to treat students in our classes as if they’re a monolith. In particular, there’s a powerful collective story about who goes to college and why. And many of us have unconsciously absorbed this story about who goes to college and why and it does not relate to reality. It doesn’t reflect the reality of who’s in our classes. So a big part of what we need to do is understand how to make visible the rich complexity of the stories of learners in our classroom. So my advice is to start with the things you know, which is, if this is something that’s new to you, the very first thing I would say is just make space and provide value for building relationships in your class. So by that, I mean like devote some time where you’re building relationships, where students are building relationships with one another, and put some value on that. So if the currency in your classroom is points, make some points that are associated with building relationships so you’re communicating that this is actually a highly valued part of the learning. And if you’ve already done that, then I should say the second step would be thinking about structure and transparency. So building structures that are clear and transparent for students. So the transparent syllabus and assignments are a great way to start with that, the idea of making what’s hidden, visible for students, and that helps us unpack those stories, because that collective story that many of us have absorbed is the students that are coming to college already know what it means to go to college and for many of our students, that’s not true. So helping make visible what’s hidden. And then the third thing I would say, which is like a thread throughout the whole book, which is grab a friend or some friends and sit down and have some conversations about it, get a book, read it together, but find some partners in crime in here to help you figure out how how you’re gonna teach towards equity and what it might mean for you to teach toward equity. So you can find some really firm grounding and footing for that.

Rebecca: One of the things I really like about how you’re describing teaching toward equity is that it’s a spectrum. And that it’s not equitable or not, but you’re teaching towards it, or you’re moving in that direction, or you’re pushing the needle there. And I think that’s a much more palatable approach than something that feels absolute. And we all know, it’s not actually absolute anyways.

Karynne: Yeah, I think that’s actually woven throughout the book. We really try to encourage folks to take the smallest step regarding anything. And then we also very much encourage collaboration. So find a friend to do this with somebody who’s like minded, and you’re never going to get there. So we’re not there. But this mindset that you’re moving in that direction is really helpful. And I think that’s, like I said, woven throughout the book.

Cynthia: I just think so often, when we think about equity, we think of it sometimes only in terms of content, like the authors I’m teaching, the scientists I’m including, and so forth. But we also like to think about equity in terms of the strategies, not just the what we teach, but the how we teach. And I think oftentimes, that’s an area of equity that people haven’t thought that much about.

Rebecca: Those are all really good points.

John: So one area where perhaps there might be some inequities is in terms of class discussions, because some students would like to talk all the time, other students are a bit more cautious, and sometimes even think about what they want to say before they say anything. What would you suggest to create a more equitable environment for discussions.

JULIA: I’ll jump in and say, my favorite for this, and first, I would say practice in very low risk [LAUGHTER] situations. First is the circle of voices. So this idea that you’re moving around in a circle, and everybody has a chance to speak uninterrupted, so that you’ve lowered the barrier to entry, and that you’re practicing this regularly, so that every student has a lived embodied experience of what it feels like to speak before you let go of those structures, then they’re much more likely to engage once they’ve had that kind of an experience. And then any kind of structured protocol where students are not spending their cognitive power, trying to think about how they’re gonna navigate the space, because it’s really clear how to navigate the space. So they can think about the ideas and do deep listening.

Karynne: Another that we all tend to use, is having community agreements. And we’ll probably talk more about that. But going through that experience with learners, and saying, “This is what we are committing to, and this is what we will abide by.” And that way, those for whom it’s just really, really difficult to speak in a large group won’t feel put upon to do that. If your community says we’re gonna encourage people, but we’re not going to require that or we’re going to ask people to be mindful of how much they are speaking, but we’re not going to close them off if they feel the need to say a second thing.

Caralyn: And I say that sometimes we think about discussions, and we just envision like, okay, we’re all sitting around a table having a classroom discussion, but opening it up, thinking about Universal Design for Learning, and that multiple ways for students to express themselves. So maybe it is an online forum, or maybe one is this synchronous or asynchronous, so that it’s not a, okay, you need to get up and speak in front of 20 people, but maybe you get some time to write. And here’s where the pre-flection questions can really help too, because having some time to think and write beforehand can make for such a richer discussion.

Rebecca: I think the pre-flection also offers that opportunity to transition into a space. You’ve been in this other place, or I was at lunch, or I had this thing, or I had this other conflict on my mind. But then here’s some time to get in this space of what it is that we’re talking about, which does allow people to focus more. So you also advocate for a strategy of “dilemma-issue-question.” Can you talk a little bit about what this is and how it’s a useful strategy?

Caralyn: The dilemmas issues questions, or DIQ approach is basically a framework or a model for putting the course content or the skills that you’re helping students master into a big framing question or a societal issue that students care about. Because we need to provide the “why” we need to provide the “here’s the purpose,” the reason for learning this. So if I’m teaching evolution by natural selection, rather than just diving into “here are the criteria,” maybe pose the question of “which species will be able to evolve in response to climate change?” because now we care about learning about what do we need to know to be able to answer that important question. It helps students connect. It’s an equitable practice because they’re bringing their own lived experiences. They can see where the knowledge and skills are useful, and they get to be creative and do creative thinking, critical thinking, and it’s so much more interesting and fun to teach. You can just take it in so many different ways. And we don’t have to look too far outside of our ivory towers to see big societal issues that we’re all going to be facing, especially many of our students. And if we want to have hope for those things getting solved, then I think providing students with that sort of training and modeling that in the classroom is just so important.

Karynne: Not just the importance of doing this, but really changing your mindset about what is important content in your class. We’ve done a lot of work with other faculty on the content doesn’t have to be these 9 million things that you’re going to be tested on at the end of your chemistry degree, but rather, this ability to think in the present and in the future and solve problems that really, really matter. Hence, learning that matters. I think that’s important to point out. So I think it’s a jazzy name that we’ve come up with, for a dilemma issue and question, DIQs. But I also think this mindset is just so important to develop.

John: And students, I think, would find it easier to learn about things that they care about, where they see the intrinsic value of what they’re working on.

Caralyn: Yeah, because that’s all of us. I feel like every other article in the Chronicle or Faculty Focus, it’s like,” Oh, do we have a student engagement crisis?” And it’s like, “okay, well, how do we engage people?” We engage them by having things that they’re interested in and passionate about, and find purpose in, and that’s where you can have projects where students, they’ll blow you away with what they’re doing, and how much work and time they’ll put into it because they care.

John: One of the issues that people have been complaining about for the last couple of years, since we move back to face-to-face instruction is what appears to be a lack of student motivation. So one way of addressing it is asking students to work on things that they find interesting, and that they can see the value of. Are there any other strategies to increase student engagement and motivation?

Cynthia: Well, I want to start by saying that I really think the decline is real. When I’ve been at national conferences and just talking to faculty from all over, it just seems like it’s what is on everyone’s mind. I have absolutely seen it. It’s interesting to think about why is there this decline? Some of the students I’ve talked to have said, it’s just really hard for them to pay attention for such long stretches of time, when they got used to only paying attention, maybe for short periods of time, I think some began to question the importance of learning at all, especially in high schools. There were often times where teachers were told if the students do anything at all, pass them. And so what message does that send to our students? But a couple of really interesting things I’ve heard from students recently. One student said to me, sometimes I don’t think you professors recognize that these cutesy assignments you give us aren’t really preparing us for the future. And so I feel like anything that helps students better face uncertainty, deal with authentic problems, as opposed to ones that we’ve kind of created in the classroom. Those make a really big difference. And then, of course, some of my graduate students told me this. They had been undergraduates when the pandemic hit, they said, during the pandemic, we learned to cheat, we learn to cheat well. They were just right up front about it. And these are excellent students. And now we’ve of course, got ChatGPT, which makes it even easier if you want to cheat. And that’s something I’ve been studying a lot. And through studying chatGPT, oddly where I came out, after weeks and weeks of study was that students valuing the learning Is everything. Students, valuing the learning is everything. It’s the answer. It’s the foundational answer. And so the learning must matter. And so I’ve been thinking a lot about what we know about intrinsic motivation, and what makes someone value learning. They value learning when they have more autonomy, how can we increase autonomy? They value learning more when they feel a sense of mastery over what they’re learning. They value learning more when they see the purpose. And often the relationship-rich type of classroom also makes them value learning more. So every once in a while, I think, would we have written a different book if we’d written it post pandemic? If we’ve written it post ChatGPT? And I think the answer is no, I think we would have written the same book, because everything in the book is geared toward that type of teaching and learning that is so focused on intrinsic motivation and engagement and relationship building and connecting to the world beyond the classroom. It’s almost like we saw this stuff coming. I don’t know. [LAUGHTER]

Rebecca: I had an interesting conversation with a colleague yesterday about a conversation she had had with some graduate students that talked about why the students were in graduate school. And they said, “Well, I kind of got cheated out of my undergrad. I didn’t get the undergrad experience because of the pandemic.” And so the motivations that we might assume, that are not necessarily real, of why someone’s in school in the first place, it was just kind of interesting to hear the perspective that they’re not here necessarily to get a particular kind of experience, they feel like they didn’t get. So finding a way to get them to value the learning is really important. And knowing they’re not here because they are motivated, because they’re so excited about a particular topic, which you might expect of a graduate student, I think is really an interesting insight to consider.

Caralyn: And I think it connects back to what Julia was saying, we need to know our students in order to understand: What are their motivations? Why are they here? …and we can’t just assume that they’re coming in with the same reasons we did. We need to take the time, build the time into our courses, to get to know students and have those relationships.

JULIA: I’d also add, if we’re really serious about making students the agents of their own education, we really need to look at the structures of how our institutions are set up, because they’re just so patronizing in every way. Like when students come, there’s so many ways in which they get messages about how they are not able to make decisions about what they can and cannot do that the institution, the professor, that they all know best, and that they need to fit themselves in the mold of that. And that mold is often defined by that story I was talking about earlier, that one story about who goes to college and why? And there’s a lot of unlearning we need to do in higher education to create institutions that actually center student agency.

Rebecca: So we know that institutional change isn’t fast, and requires a lot of people to push against the current structure to change the structure. And one of the ways we can do that is thinking about our own courses, a place that we do have control over. So can you talk a little bit about some of the strategies we can encourage faculty to adopt or practice in the spaces they do have control over, that would help us move into this transformative space and move towards equity.

JULIA: One of the ways in which we do this in the book is we do it in thinking about designing learning experiences from a liberatory framework. And I’ll back up and say backward design has been a really valuable tool in faculty development and teacher preparation, and really has helped change the way in which we think about how we teach. So instead of allowing a textbook or some other driving force, determine what the order is, and the pathway for teaching, we’ve thought backwards about what are the goals we want. One of the challenges with that, I think, leans into what I was just saying is that centers, the faculty member’s thinking very much. It’s a faculty centered thinking design process. And so something that we really tried to do in the book is think about how might you decenter the faculty member in that process some, so that you can bring in some of those student perspectives. And so we did this in a couple of ways. I won’t talk about all of them, because it would take a really long time, but one I want to mention is using design thinking as an approach to complicate that backward design process. So design thinking is an approach that we borrowed, not just us, but lots of folks in higher education now are borrowing from product and software design. And design thinking really starts with centering the user of the design. So if you were a product designer, you would start by trying to empathize with the user. So for example, if you are a toy designer, you’d want to observe children in play and engaging with toys to understand how they engage with toys. You might also want to dig into some research about child development in your target age group so that you could think about developing that toy to be appropriately developmental. And so we translate this in an activity in our book using an empathy map. And the way that we did this, which I think is quite powerful, is we built some composite student personas that tell different stories about students in college, but they’re based on data. To build these I use the institutional data from our institution. And also if you’re familiar with Jeffrey Selingo’s student segments, we use those as well to build, I think there are five of them, that tell different stories about students in college, their histories or herstories, and also their goals for being in college. And then the exercise asked the educators to center themselves in that narrative, and think about what kind of messages might that student be getting from their family, from the college, from the society at large? What kind of goals might they have for themselves, and really think deeply about these before you write your learning goals and decide what activities you’re going to do and set up your learning environment so that process of backward design can really be influenced by having a deeper understanding of the types of students that are actually in your classroom. I’ll just say with a caveat, these student personas were derived from our data and every institution is different. So it really helps to make your own. And from the concept of design thinking, the best approach is to have access to the actual users, which is not always practical in higher education. But another way you might do this is to interview students who’ve been through your class to think about a redesign, you might interview them to understand how they engage with the material. And this is a great way to use an assessment technique through an empathy map.

Karynne: Could I add a couple of things? One is about the process and why we liked this design process. And that is iterative. And the more I talk, the more I’ve realized, oh, everything is just iterative. And so I really liked that we get to embrace that and realize that, okay, it’ll be different next time, it may get closer to the mark if I do this. The other thing I was going to comment on is, and we’ve all done this as well… So it’s not always practical to design the course, but sometimes co-designing with students is really, really powerful. And we’ve tried to take advantage of that when we have that opportunity, just again to send that message, like it’s not about the professor’s experience, it’s about the learners’ experience.

JULIA: Even taking little pieces and co-designing them… I taught a general chemistry class for years and years. And I had a rubric for the final grade and we just co-designed that every year. But it was the only thing we co-designed because we didn’t have time to do the whole course. But that was a pretty powerful thing to co-design at the beginning.

Rebecca: As a designer, I appreciate everybody talking about design thinking. [LAUGHTER]

Caralyn: It took us a while to get there when we think about higher ed, but it makes so much sense. Who do we really want to be thinking about? …and it’s the learner and their experience.

Cynthia: And I often think about the who that we’re designing for, and that all too often novice professors, I find, tend to design for a younger version of themselves. Older professors tend to design for kind of an average student. And then every once in a while someone is designing for an anomaly, where they had a student a previous semester who did something terrible, and now they’re redesigning the course ao that never happens again. And I think any one of those can be problematic, and that we’re often better off trying to design with a variety of students in mind, and not just a single concept.

Rebecca: You mean, we don’t have just one student?

Cynthia: it turns out, we don’t. [LAUGHTER] ibut that would be nice.

Rebecca: It would be a lot easier.

JULIA:Getting specific is important here. The generalities are, I think, the problem. And so what the personas do is they provide some really specific cases to think towards. So you’re not thinking in general about a group of people that morph together, but you’ve got like, one of them is Juwan. And he’s a military veteran, and he can only go to school part time, and he needs to work two days a week. Just getting those details in your mind when you’re thinking about the design are really, really valuable.

John: Might it be helpful also, to get data from your specific students? Do a survey of them asking about their life experiences, about what has worked well for them in the past, and what challenges they’ve had in prior classes or where things didn’t work so well, so that you can address some of those in designing your course, perhaps co-designing, or at least responding to, the students expressed concerns.

Caralyn: There’s so much information there, and it helps going back to building those relationships, they want to be able to talk about who they are, especially if they see that you’re responding to their feedback and changing something because of it. That models such awesome behavior.

John: And if you know some of the things your students are interested in, you can use that sometimes to design activities that may appeal to the specific mix of students you have in your class. So you’re not teaching to that generic student, you’re working with the actual students in your class,

Cynthia: You could design even the assessments around those students sitting in front of you.

Rebecca: What? [LAUGHTER] Tell me more about that.

Caralyn: When we get into assessment, and this is where, when we were writing and that was this collaborative writing process, where I learned so much from Julia, Karynne, and Cynthia about this, and I feel like assessment is the area where there’s so much I can do, personally in my own courses, but also where I look at like that’s where we can have some of the biggest impact because I think our assessment practices have not been well designed and we have done harm and we need to fix that. And I think we advocate for connected assessment. So assessments where they are aligned with learning outcomes, of course, but also working and designing for the whole student. So they’re holistic, they’re affirming, so we’re not trying to be punitive. We’re not trying to like here, let’s go in looking for those mistakes. But we’re looking at, “Hey, where’s the growth happening? Where’s the learning?” …and highlighting that, and being so much more focused on giving feedback and process, so, “Here’s how you’re going and here’s how you move forward” and not just like, “Okay, here’s the percentage and you should know what to do with that,” because it turns out most of us don’t. And being able to have authentic experiences, and the end, like was mentioned earlier, being really transparent. Having examples, having models, being really clear about “here are the steps.” Because if we have, “okay, here’s a project, you’re going to write a lab report,” but I don’t describe actually what goes into that, and what are the steps in how to do it, well, then I really shouldn’t be surprised when the final products are not awesome, because I didn’t provide enough scaffolding to get there. And this is someplace where I’m still doing a lot of work here thinking about my values in teaching and how if I’m looking at that, now, for me, it might be that reading table on the syllabus, like here’s where the points are, here’s where things are coming from. How does that align with my values? How does that align with the message that I want to send students? And where we can being as intentional there as possible, and talking to students about what is the message they’re getting, because what I am intending might not be what is being communicated. And then where we can, really thinking about and being open to taking a risk with some alternative grading strategies. Maybe it’s ungrading, maybe it’s specifications grading, but there are so many more resources and great smart people doing so much work in this area. And every single one I’ve ever talked to or reached out to is always super excited and willing to share their ideas and share what worked and what didn’t, because it can just really change the entire feeling in a classroom when we take away the power of grades, because they’ve really been used to stop learning and oppress in many cases. And if we get rid of that, it really opens up the space for some honest relationships.

Cynthia: Unfortunately, you have to end a book at a certain point and publish it, it turns out, and one of the things that we didn’t really get to talk a lot in the book about was ungrading. We got more into it right after the book came out. But that’s where having a website that goes along with the book has been such a great help, because we were able to put so many fantastic resources about ungraving or minimizing grades on that website. And that made me feel a lot better. Because for myself personally, getting involved with ungrading has been one of the most important things I’ve ever done for my teaching. No one told me it was going to change everything. [LAUGHTER] I thought it was just going to change one little piece, but it changed everything.

Karynne: One of the things that I’ve tried to do with the ungrading is to share with learners… mine’s a view, it’s not the only view… and I never want to be punitive with grading. If you feel like I’m punishing you with grades, please, we need to talk so that I can know more about your assets, know more about your desires, and help you head in the right direction not punish you because you don’t know something. It’s like that’s what learning is. And so that’s just been a practice of mine.

John: We’ll share a link to the website for the book in the show notes so that people can explore some of the additional resources there. One of the things you advocate throughout the book is the use of active learning approaches. But you also note that you should probably expect some pushback from students. What are some of the most effective ways of addressing the pushback from students who prefer learning by being lectured at so they can sit there passively without having to actively think about the content?

Karynne: One strategy actually you can use is to be upfront about it. So students in this course before when I’ve used these things, some of them really don’t like it, they’re very uncomfortable. So I just want to tell you that I’m aware of that. And that’s actually a point where I bring in that idea that I don’t wish to be punitive regarding assessment, you’re going to have as much say in this as I do. So the other thing is to share the the literature and the research. And again, since I primarily teach people who are going to be teachers, they really need to know about what the literature has to say, what the research has to say about learning. And it occurs when there’s some space between absolute comfort and absolute chaos or uncertainty. There’s going to be some uncertainty, so we always try to share that with learners as well so that they can go back and tap into that research. Another thing that we really try to do is to use self assessment and reflection as much as possible, so that you’re letting us know where are the ways that you are growing. I may not be aware of all the things that are changing in you and if you are able to inform me of that, that’s a much more informative approach, then, okay, I’m going to do all of the assessment. We had to learn this ourselves, [LAUGHTER] to start expecting the push back. And then they think that you don’t like them, because you’re not teaching the way that they prefer. And emphasizing that collective, how we’re all changing, how we’re all growing here, I think is another approach that you can take.

JULIA: I would also add just tapping into their lived experience of learning something new. And often they can really embody that if it’s not something about school. So like, are you good at tennis? What did it feel like when you first picked up a racket? Or did you try and learn something and give up on it? Why was that? So understanding that actually getting really good at something does have this period of discomfort before it becomes a regular part of your life so that you understand that that is actually getting you to a point where you’re going to be a different person and transformed.

Cynthia: A friend of mine noted that her students at first they said their fear was that the assessments weren’t going to match the activities in class. And so that made me think, oh, that’s probably something I need to say right up front is, this is what the assessments are going to look like. Here’s how what we’re doing in class is going to feed into that, because I can see where there may have been professors they had in the past, who taught in a way that was very active, but then assessed in a way that was very passive, and students might have had trouble making the match.

Rebecca: Well, there’s been so much great insight in this conversation today. So thank you so much for that. We always wrap up by asking: hat’s next?

JULIA: Well, that’s an exciting question for us. And I actually want to start by talking a little bit about how we ended the book, because a thread in the book that we haven’t talked too much about is really focusing on the educators identity development as an educator being a really critical piece of this whole journey. So what you’re putting together for students and doing for students may often feel like it’s just all work that’s flowing out of you. But also a very important part of that is your own development over time. And so our last chapter is called “Your turn, self and collective efficacy,” and it was really important to us to end by saying to educators, it’s important to think about who you are as an educator, and invest in yourself that way. So that’s one thing that I just wanted to put out there and make sure that people understood that that was a value for us. And in terms of what’s next for us is we are really, really, really excited about launching a course design institute that’s based on the book, which we’re going to host in August, it’s August 4th to 7th, it’s called “Learning that Matters: the Course Design Institute.”Iit’ll be here in Olympia, Washington, a really lovely place to be in August. And we’re at Evergreen State College, which is a College in the Woods, very beautiful. We have a farm and a beach. And, [LAUGHTER] I know, we’re very lucky. But the idea is to have an immersive collaborative environment to design or redesign the courses that you’re going to teach in the next fall. And to do that with people who are not necessarily at your institution. So to get a variety of voices and feedbacks. We’ll have a lot of time for you to work on your own, but also a lot of time to talk with people from different kinds of institutions who are working on different kinds of problems, teaching different kinds of courses, to build that interdisciplinary approach to the work that you are doing in your classroom and also help you build a wider community. So this is something we’re super, super excited about. And we will share that link with you so you can put it in the show notes. And then as a little bit of a teaser, we’re doing a free virtual workshop on May 9, this is at nine o’clock Pacific Time, which is noon Eastern time. It’s called Making Courses Memorable Beginning and Ending and I’m not going to say more about it because I want your curiosity be sparked there.

Cynthia: And of course, we’re also always happy to zoom in with people who are using the book for book clubs.

Rebecca: Well, thank you so much for joining us. We look forward to sharing this and encouraging folks to pick up your book.

Caralyn: Thank you.

JULIA: Thank you.

Karynne: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

281. The New Science of Learning

Students who enter college without a preparation in effective learning strategies often do not persist to degree completion. In this episode, Todd Zakrajsek joins us to discuss what incoming students should know to successfully navigate the college experience.

Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of 4 Lilly Conferences On Evidence-Based Teaching and Learning. Todd is the author of many superb books. His most recent book is the 3rd edition of The New Science of Learning: How to Learn in Harmony With Your Brain.

Show Notes

Transcript

John: Students who enter college without a preparation in effective learning strategies often do not persist to degree completion. In this episode, we discuss what incoming students should know to successfully navigate the college experience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Todd Zakrajsek. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the Director of 4 Lilly Conferences On Evidence-Based Teaching and Learning. Todd is the author of many superb books. His most recent book is the 3rd edition of The New Science of Learning: How to Learn in Harmony With Your Brain. Welcome back, Todd.

Todd: Well, thank you so much. I’m looking forward to our conversation today.

Rebecca: Today’s teas are:… Todd, I hear you have a surprise for us.

Todd: Yeah, actually, I’ve got a bag of mystery tea. There’s just a whole bunch of different teas in here and they’re little packets. So live in an air we shall open up one of the packets.

Rebecca: So, would you like a drumroll? [LAUGHTER]

Todd: There we go. And now I am going to be having crystal clarity oolong tea to find a peaceful state of mind. Nice.

Rebecca: Sounds like a good state of mind to be in.

John: So it’s a fermented tea.

Todd: Apparently it is.

John: Where’s that from?

Todd: This is from Portland, Oregon.

John: Excellent. And I have a blueberry green tea from the Republic of Tea.

Rebecca: And John, just because you were asking about it last time we recorded, I have my last cup of Hunan jig, just for you.

John: Very good.

Rebecca: I do not know why it’s called Hunan jig. [LAUGHTER] It’s tasty.

John: We’ve invited you here today to discuss the third edition of The New Science of Learning.

Rebecca: Can you give us a little overview of the book?

Todd: So the book is essentially a guide to help faculty and students to understand the learning process, but also just the whole college experience. Now, this is not a book that’s like tips on how to study, specifically, it’s more of a global looking at teaching and learning. It does have tips in there too, and actually each chapter has a couple, but that’s not the foundation of what I’m really after. For instance, the first chapter is about learning from multiple perspectives. And it talks about the dangers of dichotomous thinking. Too much in our society, it’s either I like it, I don’t like it, that’s a good person, bad person, and gets away from that. And from there, there’s sections in there on setting goals and self regulation, monitoring how we interact with others in our work and self efficacy, the extent to which we believe we can succeed at something. There are whole sections on helping to understand how people learn, finding patterns, and what that does in our society, in our classes, and in our content. If we can find the patterns, we can learn a lot more easily… Bloom’s taxonomy, and it has a chapter in there on sleeping, the effects of not sleeping or how much it can help you when you do sleep and exercise. And it even has a chapter in there about how to work well in a group. So it’s essentially kind of an overall book that helps students with the learning process or the college experience.

John: The book is clearly a good resource for first-year students, that said books often have more than one audience. Did you write this book for a broader audience or was it focused primarily on first-year students?

Todd: It’s always tricky writing a book, in my mind, at least you have to have your audience in mind the whole time you’re writing, that’s the only way I can do it, and write at a level that I think connects with the audience. So certainly, first-year students and more specifically, like a first-generation college student, or a student from a marginalized group that doesn’t have a lot of experience in their family with colleges. Because if you do, it’s a very different experience than if you don’t. So this is a resource for people who don’t know the ins and outs. At the same time, there’s a lot of material in here that faculty just don’t know. And so some of the learning theories that are in here, some of the pattern recognition, some of the sleep research, the faculty don’t know. So I tried to write it so that faculty would also find it interesting. And I tried to straddle that line, but I also tried to pull in what a senior in high school might find valuable. So a junior or senior in high school could read this and get a better sense of what they were going to experience in college. So I tried to do that, and then a general resource for anybody else in terms of people in student affairs or in a student success center. So I was looking at multiple audiences started primarily with the student. But when I used examples and the level of writing, I tried to drift in and out so that I could get these other groups in such a way that they would find it valuable as well.

Rebecca: I really enjoy the personalized conversational tone, which obviously is great for students. It hooks you right in and then goes into the introduction. And so I really enjoyed that style. Can you talk a little bit more about why you chose that style and how that might help students?

Todd: For me, I just think conversation storytelling is one of the most effective ways of learning and so I like to do that. I also like to bury things just a very little bit at the beginning. So you start to read and then you realize as its unfolding… So, for instance, in the first chapter, we’re talking about the danger of dichotomous thinking. The example in the book was: it’s easy to tell night from day. If it’s noon, we know it’s day and if it’s midnight, we know it’s night. But what happens just as the sun sets? There is a moment when the day stops and the night starts and it’s the edges like that where all the richness is. And then I think there’s a line in the book that says, once we’re going through that, like the day and the night and what’s really at the edge, and it’s like, we’re not really talking about day and night here, are we? We’re talking about people. And so that kind of concept that I really like in terms of keeping it conversational, keeping it tied to things that people know. But my whole goal, and what I’m shooting for is to help people who can read science, but do it in a way that they enjoy it.

John: Much of your book focuses on how we learn. Students come in with some serious misperceptions about how we learn. When students are asked how they study, they tend to read things repeatedly, where the evidence suggests that’s not very effective. They tend to highlight quite a bit, which is also not very effective in increasing long term-recall, or transfer ability. Why aren’t students learning how to learn before they’re, say 18 or 19 years old or older? Shouldn’t some of this instruction be taking place in earlier years of education? And why isn’t this happening earlier?

Rebecca: John, did you bring your soapbox with you today? [LAUGHTER]

Todd: It’s an issue. I mean, it’s a huge issue. We spend our whole time in our educational system teaching people stuff, how to do things and what things are. But it’s crazy, we don’t teach students how to learn. We treat it as if it’s an implicit assumption that everybody just can learn and to an extent we can. When we’re young, we learn how to walk and we learn how to use utensils, and we teach kids how to tie shoes, and children learn how to ride a bike. And so I think in the general framework, there’s all this learning going on and teaching going on. And as a result, we have the implicit assumption that everybody can teach and everybody can learn. But teaching is a profession. And learning is really, really nuanced in a lot of different ways. And so what we have are a lot of implicit assumptions and trials and errors. If you think about for yourself, where did you actually learn how to learn? And for most individuals, it’s around second or third grade, because that’s when we start testing. Which by the way, if you ask young, young children: “Do you like to learn and do you like school?” they say yes to both of those, until suddenly, they start to say that they like to learn and they don’t like school. And it’s almost universally in the country around third grade. So right around third grade, we’re starting to test, but we don’t teach them how to learn at that moment. So the parents are making up flashcards and quizzing the kids and the kids are reading aloud in class. And we’re going through these actions without knowing what we’re doing. And it turns out, as you’ve already pointed out, John, very well is that a lot of these things that we have implicit assumptions about, we’re wrong, we’re just wrong. And the trouble is, we don’t have a baseline. So if we start highlighting, if I highlight the chapter in the book and I get a good grade, then obviously highlighting must work. And if I’m underlining things, and I get a decent grade, underlining must work. But how much could you have learned if you realized how to do that? Now I’ve got students who will use five different colors to highlight. When I ask them why the different colors, they’ll say, the blue is if it’s an application, and the green is for vocabulary, and the pink is something that’s like really important, and I always love to tell them when they do that. You should use black for the stuff that’s not important at all. Good times. But the idea is they’re doing that the students sometimes are doing their flashcards but the question becomes… like flashcards, when you’re going through a deck of flashcards, when you get it right, do you set it off to the side? Or do you put it back in the deck? How do you do that when you’re learning something like chemistry? How do you learn those terms? When you’re learning a periodic chart, how do you do that? And so I just firmly believe if we started teaching children how to learn at around second or third grade and just spent 1% of the time teaching the learning part and the rest of it all about content, by the time that they were done with school, they would be lifelong effective learners. And instead, we have people who believe that they have a given learning style, which we could do on a whole different show. People do have different ways in which they learn, but the concept of teaching to a given learning style has no data behind it. And highlighting, there’s studies out there that says highlighting doesn’t work and it doesn’t work primarily, however, there are ways to highlight that are effective. Re-reading is not effective, unless you reread for a specific purpose or reread in a special way. And so this stuff is going on, and, again, we’re not teaching students how to learn. So we should do that. I’ve given presentations about how to learn to everyone from high school students up through professional schools, in nursing programs, pharmacy programs, and medical schools. And the number of times that someone in a medical school… a second-year medical student… will come up and say I wish somebody had taught me this sooner. The case you’d think of is like the prototype of a student would be a med student who memorizes and learns stuff so fast. But those students who can pick up things quickly will say I wish somebody had showed me how to pick it up even faster. And so I think we should do that. Same with writing. We should teach students how to write instead of just having them write. There’s a soapbox for you. [LAUGHTER]

Rebecca: I mean, I’m on it, too. [LAUGHTER] It’s interesting that when I was thinking about your question and about like, when would I say that I started learning was actually when I started struggling.

Todd: Oh, interesting.

Rebecca: …because when it wasn’t hard, you could just skate by, but there was a moment, and it was in sixth grade, I remember, social studies, and I had a really hard time reading and reading comprehension. And then I had someone who actually had to read more effectively. And it worked immensely. But it was only because I had that intervention or that help. Because it wasn’t part of the curriculum, it wasn’t taught. that I actually overcame that. But I think a lot of our students come to learning these strategies once they’re struggling significantly to the point where they have to ask for help, rather than us being proactive about it.

Todd: Exactly. And I tell you, and it’s in the book here, too, is it’s exactly what you said was my experience. I pick stuff up very, very quickly, I could skim a book and go in and take a test and do well. And I did that through high school. When I got to college, I had five classes. My first class that I got a grade back in was like a D minus in the introduction to criminal justice class. And then I had a physics class and my first grade and that was an F. And I thought, well, what’s not going well. And then in my math class, I got an F minus. And I remember thinking, well, it can’t get any worse than this, until I got my chemistry grade and that was an F minus minus, I even went to the teacher and said “F minus minus? I don’t understand this.” And he said something like, “Given you received an F minus minus, it doesn’t surprise me you failed to comprehend it.” So, a kind of mean person too. And the concept here, and the reason I mentioned is what you just said, Rebecca, I hadn’t learned how to learn. And so at the point where I needed to know how to learn, I was in a jam. And I actually went to the registrar to get a drop slip. And she said, “Get your signatures and bring it back. I’ll take care of it.” It was a long time ago, and four of my five faculty members signed the slip. This was a very small school, this isn’t some big school where you get lost. This school only had like 200 faculty members and about 3500 students and the psych prof said, “I don’t understand what you’re doing. Why would you drop out?” This was like two months after I started. And I said, “I just can’t do it. I don’t know how to do it.” He said: “You need to learn how to learn.” I said “like you can learn how to learn.” I didn’t even know the concept existed. And so he pointed out some strategies and pointed toward a book and I learned how to learn but I was one signature away from not finishing, I wouldn’t have met you, I wouldn’t have done any of the books I’ve done all of that would have not happened for one signature, because nobody taught me how to learn.

John: And a lot of our students get those last signatures and disappear. We’re losing a lot of students once they hit that barrier, which is why it’s important we have books such as yours, and we spend more time working on teaching students how to learn.

Rebecca: And reading the book as part of our system.

Todd: That’s what we should do. If I’m doing a faculty workshop at a campus, and I say “How many of you in here came within a whisper of flunking out of school?” most faculty raise their hands. And that’s just amazing to me, those are the folks that you would think got through easy. So it’s what you just said, John, how many fabulous, wonderful people, they’re probably doing things that are fine, but they’re not doing what they wanted to do and it’s because of that.

Rebecca: Yeah, we just need to design our systems to be proactive, rather than reactive. And oftentimes, it’s not even reactive, we just miss the boat entirely.

Todd: That’s a good point. Instead of being reactive, we should be either proactive, or at least not inactive.

Rebecca: Let’s start with not inactive, yeah. [LAUGHTER]

Todd: That’s a place to start.

John: Part of the issue is that we ultimately figured it out on our own. And we assume that everyone can and we’re not a random group of the population, a very large share of faculty members were not first-generation students. A disproportionately large number of faculty members come from families where there were people with higher ed is part of their background. And it’s easy to forget what sort of struggles students may face. Even if someone may have come close at one point, they figure it was an aberration, and they forget that those aberrations can be critical points for many people.

Rebecca: And that these struggles happen across the spectrum. It’s not just our undergraduate students. As you mentioned, our graduate students have some of the same struggles. I was just having a conversation with graduate students last week about even just basic time management skills or how to troubleshoot or problem solve, because they don’t have those skills, and they need to build those skills.

Todd: Yeah. And it’s still also not equitable across different groups, individuals from marginalized groups tend to fail more frequently, because they don’t have the resources and they don’t have that support system so that when they are struggling, somebody can help them.

Rebecca: So the first edition of this book was released in 2013. How does this third edition differ from these earlier editions?

Todd: Actually, in a lot of ways. When we wrote the book in 2013, first of all, the research has changed considerably. But the book ended up also being just a hair over 100 pages. And this new version is about 250-260 pages. So it has grown substantially. There’s sections of the book that were not in the original book, or not even in the second edition. So there was a whole section on how to learn in groups. There’s a section in there on pitfalls, the places where students tend to have problems. Hidden curriculum kind of issues: what are things that they’re not specifically stated and so they’re implied in a way that if you know that they exist or you had family members who went to college you can figure it out. But if you’ve never gone to college, you didn’t know, I didn’t know when I went to college that if you failed a class, you could retake the course later. And so I thought when I failed my chemistry class that I was literally done, because if you can’t pass, then I can’t get into Chem II. If I can’t get into Chem II, I can’t go… And when I talked to my advisor, the advisor says: “Well, just take a trailer course.” And I said, “What is this thing you call a trailer course?” So those types of things are in this edition of the book. So I picked up a lot more nuances than we had before. And of course, I mentioned a little bit earlier, too, but the research has changed significantly in the last 10 years. We know a lot more now about how we learned than we did 10 years ago. And for things as subtle as what’s happening while you sleep. And so it’s getting more and more that we know actually, what kinds of learning is being solidified at different stages of sleep. So there’s always changing research, and I’m just happy to be able to get that updated research in there.

Rebecca: I love that you just slipped in something about sleep, because I was just going to ask about sleep, I was just having a conversation with a colleague today about being able to process new information when you’re tired. And that we might typically think of processing being associated with a learning disability or something. But actually, lack of sleep can cause the same kinds of symptoms, essentially. And so I can imagine that actually talking about sleep as an easy sell for students, because it’s something that everyone can easily think about, but many of them don’t get. Can you share a little bit of insight into sleep and learning?

Todd: Certainly, and this is one of those areas that we all know that it’s harder when you’re exhausted to do something than when you’re rested. But back to the dichotomous thinking, we think oftentimes in terms of I’m exhausted and I’m rested. But what about all those nuances in between. What if you normally like to get seven and a half, eight hours of sleep, and you get six and a half hours. You feel okay, but what we know now from the way people learn is that you’re still going to be learning at a less effective level. And if you’re exhausted, you get to a point very easily where you can’t learn at all. And so we know that in terms of encoding the information, you need to be able to process the information in your environment, which happens when you get sleep. So that’s important. And we know that nobody wakes up after a terrible night of sleep and says, “Whew, I feel great, I look great, and I’m learning like crazy.” We know it’s going to be a rough day. And so that fatigue makes it hard to learn. And then what we also know about sleep, which is fascinating to me is while you sleep, a lot of consolidation happens, it’s called consolidation. And if it doesn’t happen, the information is gone, typically in about 24 or 48 hours. And so what we have are students who, for instance, will study all night and they can go in and take the test. They do okay on the test, so they think this is an okay way to learn. And then they don’t realize that the material is pretty well gone in two days… three max. And then later when they need the material, if let’s just say for the comprehensive final, the instructor says “Oh, everybody, I hope you’re studying because this final is going to be tough.” Now I go to learn for the final… I flunk the final, most students don’t say “Oh, I’ll bet that’s because when I learned I didn’t get stage four sleep, which consolidated the information and therefore made it available for me to relearn it at a faster pace for the final.”No, they come back with a “Wow, that was a really hard final.” So it’s going on all the time. But sleep is probably right up there at the top of one of the things you can do to learn more effectively is to sleep well.

Rebecca: Yet, so many of our students don’t sleep. And we inevitably are probably teaching a class full of students who haven’t had a lot of sleep.[LAUGHTER]

Todd: Yeah, and it’s for the public service announcement, we got to put it out there because the sleep is important in terms of learning. But there are so many things that are tied to lack of sleep, it’s just incredible: diabetes, even cancer, weight gain, high blood pressure, all these things. There’s just tons of stuff. Your skin actually looks worse. There’s so many things that are tied to a good night of sleep. It’s when all the restorative stuff happens. So I’m going to tell you listeners, the folks who say, “Yeah, I know, I’m exhausted, I can’t get my sleep.” It’s damaging to a person to not get sleep. And when somebody says “Well, yeah, but I got so much to do,” just keep in mind that it will take a toll. And oftentimes, and this is an important one, if you get a rest or get some extra sleep, you’ll do other things so much more effectively, that you come out ahead and don’t have the health issues.

John: And this is really important to convey to students. And I do share this information with students in my classes. I don’t always practice it myself, unfortunately. But I do share the information. And when they see results on how much more they recall when they’re well rested, at least a claim it will have a bit of an effect on them in the future. But one of the things… this is more on the faculty side rather than the student side… but so many of our classes are designed in such a way so that faculty are using high-stakes exams. Students have a lot of incentive to cram the night before a test and it does have that immediate payoff of increasing their short-term recall. And then, since they’re worried about the grade, they don’t necessarily care about how much they recall until they get to their next high-stakes activity. And then they have to go through the whole process again. And maybe this is something that faculty should work on too in terms of reducing the number of high stakes activities, reducing the incentives for students to cram and to cut back on their sleep.

Todd: Yeah, I think that’s a really good point. In fact, there’s several things that we can do to impact the student’s sleep. When I mention the importance of sleep to faculty at times, they’ll say, “Well, I can’t make them sleep.” And oftentimes, my response is “No, but you can keep them up.” If you have a high-stakes exam, and it’s like a midterm and a final, it’s human behavior, people are going to wait toward the end to do it. I know, there’s some faculty out there listening who say “I do everything early.” And that’s great. But I can tell you, I’ve been on a lot of committees with my colleagues, where we turned reports in at the very last minute or somebody handed me their portion at the last minute. So it is going to happen, if we know students are going to wait toward the last minute to do it. It’s what you just said John, it’s a good point, if it’s a huge exam, it means they’re going to be up, maybe even for multiple nights. If it’s a big paper, they’re probably going to spend the night all night writing it, maybe two days and it might get a little bit of sleep, but they’re going to be tired. If you have the paper due on like Monday at noon, they’ve now got exhausted from Sunday night, they’re gonna be tired all week. If you could make your paper due on Friday afternoon at like 2:00, if they stay up all Thursday night, now they’re exhausted, but they’re exhausted going into a weekend. So a lot of little things we can do. I have a friend Howard Aldrich at UNC, he had a nine o’clock class, 9 am. He had the papers due at class time in the morning, then he and I were chatting and with Sakai, you can see what times papers are turned in. So we were looking and the students were turning in the papers, two o’clock, three o’clock in the morning, even eight o’clock in the morning. And he knew that they stayed up to do it. So he changed his deadline to 9pm with a 12-hour extension, if you asked for it. So if you can get a 12 hour extension from 9pm to 9am. What he found was 86% of his students turned in the work by 9pm. So when that happens, we can’t say they were awake. But we know that they weren’t up doing his paper in the middle of the night. And so those are the kinds of things that we can do as faculty members. And I agree in terms of the high-stakes tests, we can think through what are we doing that’s actually going to be conducive to learning versus is going to make a hindrance. And if we say “Well, it’s their own fault. They shouldn’t wait till the last minute.” Why put them in that position?

John: To be fair, though, to those faculty who do give high-stakes exams, they often spend a lot of the time just lecturing in a monotone, which can facilitate sleep on the part of students, at least during their class time, which is a large share of the time that they’re interacting with students.

Todd: That’s great. Yeah, I suppose they could… get a little nap in during class. that could work.

Rebecca: It’s all about balance. [LAUGHTER]

Todd: It is balance, isn’t it? That’s a work-sleep balance right there.

Rebecca: I had an interesting conversation with students this week about perfectionism and procrastination, which also, I think leads to sleep deprivation because of all the procrastination. And what I found in the conversation… students were being really authentic and open with me… was that they were so worried about their performance on things, even low-stakes things, like these weren’t big-stakes kinds of things, but just so worried about their performance on something that they would wait to do it. But they’d spend all this energy and time worrying about it. And so we talked about how to actually take an assignment and then plan it and break it into smaller pieces. But I talked to the students about how to break it into smaller pieces so that there were times to get help, because they were so worried about not doing it well that they could build in time to get assistance and help. So I’ll be interested to find out at the end of this week, if they were going to try this strategy this week to see if it helped them. But it had never occurred to them to break it into these smaller pieces.

Todd: Yeah, and what you just said, I think, is vital for anybody who’s listening. It’s all the stuff that never occurs to somebody. This is why individuals who go to therapists can gain so much is when a therapist had some an individual says “I never thought of it like that.” For students, let’s look at your sleep. Just jot it down on a scale of one to 10 in the morning, how did you feel about how much sleep you got? To what extent did you get a good night’s sleep, and then at the end of the day, jot down on a scale of one to 10, how’d things go. And when they start to see that bad night’s of sleep result in days that are not all that productive or work well, it’s like, “Oh, I didn’t know it was that related.” Breaking things into tasks, I think is fabulous. That’s what this book is about, too, is the concept of just showing them things and then having them be able to look at and say, “Oh, I had never thought of that.” And that’s what’s valuable. So I like what you’re doing.

John: For those who procrastinate on coming up with the set of tasks to do, again, course design could resolve that a little bit by scaffolding the project so that students never have a huge chunk of work to do all at once.

Todd: Yeah, I think that’s good. And then the other one is a whole different program is ungrading. And if we can just remove some of the grading on some of these things, and there’s faculty out there and myself included years ago who would say well, “If I don’t grade it, if there’s no grade, why would the students do it?” And it turns out, sometimes, from what you just said, if the students are so stressed about it, they spend all this extra time, you remove some of the high-stakes aspects of it, and they don’t stress about it so much. But that is a problem. And I will tell you, it’s not just the students, I was writing a blog, and I have a person I turn a blog over to McKenzie. She’s phenomenal. And she edits at the end. And I wanted to prove to her I was working on this one blog, because I told her I was going to get it and and I kept getting busy with other things. And I submitted it to her. But I planned on spending another three or four hours on it. She emailed me back, and she said, “This is so close to being done. Let me just edit it. And then you can take another look at it.” Had she not said that I would have worked another probably four hours on this thing, half a day. And I think students are doing that at times too. I think they finish an assignment. It’s good. And then they think, “but I want it to be better.” And so, just clarity and helping to understand and building some structure into the course so they’re not guessing. Take away the stuff that’s just not necessary and let them focus their energy on the things that are necessary.

Rebecca: Sometimes it means even pointing out something that’s low stakes is actually low stakes.

Todd: Yeah, I think that’s really good. Signposting. So there’s a terminology for you. Signposting is basically telling somebody what they’re doing, or what you’re doing. So if I’m giving you feedback, I could just give you feedback. And we’ve had programs where at the end of the program, the students will say, “I’m not getting enough feedback.” And so we all as faculty say, all right, anytime we give feedback, we’re going to say, “Would you mind if I give you some feedback right now? Hey, would it be alright, if I give you just a little bit of feedback? Do you have some time tomorrow for some feedback?” …and at the end of the semester, the response was too much feedback. We hadn’t changed. But it’s what you just said, just let people know what’s going on.

John: One of the nice things about your book is that it’s grounded in learning science, but it’s really easy to read. One of the things we had trouble with in coming up with questions is there’s so many things that we could discuss in this book that we thought we’d shift it back to you, what are two or three pieces of advice that you would recommend to students that might have the biggest impact on their learning?

Todd: First of all, I appreciate the fact that you found it easy to read. And I have gotten that feedback from others too. It’s called “The Science of Learning.” And I think that scares some people at times. This is not a dry book, I tried to make it conversational, and folks say it’s fairly easy to get through. And that’s good. The couple things… we’ve already talked about sleep quite a bit. Sleep is just huge if we can help talk to students to sleep. The other one you had mentioned already is the cramming. The tricky spot with cramming is not necessarily that the students want to do it, they are reinforced for it. I consider this to be one of the biggest traps in higher education. Because the research suggests that if I cram all night long, don’t sleep, study all night long, and if you sleep for six or seven hours, I may very well outscore you by two or three percentage points, just enough that I do fine, and it looks like that’s okay. And you’ve already mentioned that a couple of days later, and the information is gone. The students don’t realize… sometimes they know it’s gone later. But they don’t generally know that it’s going to go away at the extent that it does. What they know is they’ve studied, they did well on the test, and therefore they’re doing okay in the class. So, a couple things in the book, if we could help them understand how much damage comes with cramming, it would be huge. In fact, it’s in the book like five times, to the point where the editor said, “Do you know you’ve already talked about this like four times?” And I said, “Yep, with any luck, we’ll only do it once more. [LAUGHTER] But it’s that important.” So that’s a big one. The other thing that I think is really huge is if we could help students with understanding metacognition, the concept here is knowing when you know or understanding your learning process, and it’s something that we don’t monitor, but we could. When you sit down to study, jot down how long you think it’s going to take you to read the chapter, when you’re done reading the chapter, jot down how you felt it went, jot down a couple of notes of what you learned. As you’re reading, stop every couple of paragraphs and just look away from the book and think, “What am I reading right now?” Because your mind will start to wander and you not realize it. Everybody that I know has read a chapter or read an article and either the next morning didn’t remember if they had read it or not or even when they finish they thought to themselves well, I don’t remember anything about that. I was thinking about bacon the entire time. And so that concept of just knowing when you’re processing… so metacognition is big, the sleeping stuff and cramming is big, and the last thing I’d say… there are lots of things in there… but just understanding Bloom’s Taxonomy, understanding at what level you know something. I like to use this as a quick example, I’m from Michigan, you could teach your students that there are five Great Lakes. Imagine they know only this, there are five Great Lakes: Huron, Ontario, Michigan, Erie, Superior… HOMES, right? And we could say Superior is the deepest. I can come back with a quiz two days later and say which of these lakes is the biggest Ontario, Michigan Erie, or Superior? And the students could say Superior. At this moment, I don’t know for sure if the students know what a lake is. I asked them these are five things called lakes. This one’s the deepest. Later I say of these things called lakes, which is the deepest?t They’ve memorized it. If students know at that moment, they’re just functioning at the recall level, ot helps them and it’s because when they take tests they start to understand I’m doing well on recall and understanding, I’m not doing well on application. So knowing Bloom’s, knowing metacognition, understanding the sleep thing, and then exercise is huge. There’s all kinds of research out there that says, if you’re actually getting your heart rate up 15-20 minutes a day, it does all kinds of cool things for your brain and actually makes learning easier. So that’s just a couple of them.

Rebecca: So in the description of your book indicate that there’s an instructor’s manual that accompanies the text. And often this is not the case. [LAUGHTER] So can you talk a little bit about what’s included in the manual?

Todd: Yeah, so when I was writing the book, the first and second editions didn’t have this. And other books of this ilk don’t tend to have it… the first-year common reads, and the first-year experience books… but I wrote an instructor’s manual when I was early career faculty member and I wrote it for an introductory psychology book. All textbooks have instructor’s manuals now, so I thought, why should this book not be just as good as those. So when I was done, I kept right on writing. And I’ve written an instructor’s manual, which, ironically, is about as long as the first edition of this book. So what I did is for each chapter, I understand that if you’re going to use this book in your classes, you may not have time to read things very, very carefully, you might have to skim a chapter at times. So each chapter has a summary. So in the instructor’s manual, that summarizes the major concepts in the chapter, every chapter has discussion questions at the end. So I put down these are the types of things that students may very well say in the discussion questions. So that if you started discussion, you’re not stuck with a situation of asking the students to discuss, you show up in class and you think, Ummm,I’m not sure what I would say about this. So I’ve given you a couple things. There’s also teaching tips in every chapter. And for each one of the teaching tips, I’ve got a short thing of these are the kinds of things that students should experience. And on top of that, every chapter also has active learning exercises. I’m big on the active learning, So it will say in the sleep chapter, here’s like four different things you can do. And it sets it all up, it explains: here’s what you tell students to do, here’s what you have them do, here’s how you report out. And so it’s kind of a guide for active learning keyed to the book. And you can find this, if you go to the Stylus Publishing website… you’re actually not going to see it unfortunately, if you go to Amazon, because that’s not where it’s listed… you have to get to the Stylus publishing site, and then you can find it and there is no charge for it, you just let them know that you’re teaching a course and they’ll send it to you. And if you can’t find it, I’m the only Todd Zakrajsek in the world. So if you send me an email at ToddZakrajsek@gmail.com, then I will make sure that I’ll get you connected to the person with the instructor’s manual because we didn’t make it real easy to find, because we didn’t necessarily think that the students should have the instructor’s manual. [LAUGHTER] So it’s kind of buried in there a bit.

John: And we’ll include a link to your email in the show notes.

Todd: Perfect.

Rebecca: So can you share one of the examples of an active learning activity that you might do in relationship to the book?

Todd: Oh, sure. The chapter on sleep, there’s one activity that’s kind of explained there for keeping a sleep activity log for a week. And it shows how to have students block off their time and then indicate whether or not things went well, or it didn’t go well for them. And it helps them to find their ideal time. So I did this when I was an undergraduate. And it was fascinating, because I found out that between 2 and 4pm, I’m practically worthless, but early early in the morning, like it’s 6 to 8am, if I do have to get up and do something, I was just really, really good. And I don’t care for getting up early in the morning, so it was unfortunate, but that’s what I found out. Another activity, and there it’s called a snowball technique. And this particular one in the chapter on sleep was students are asked to think about things that help and hinder a good night of sleep for them. And then the snowball aspect of it is they talk to other students, and then they learn one thing that helps and one thing that hinders sleep, and after you learn from five different people, you go back and sit down, you get into a small group, and then you discuss those, and then you report out kind of overall, what are the general themes that you saw. So there are things like that in the instructor’s manual, they’re described in like a half a page. So it doesn’t take you very long to read through it and get a sense of what it looks like. And so it’s there just to help you get you rolling.

Rebecca: Sounds like it really reduces some cognitive load for faculty teaching these things.

Todd: One of the issues that is tricky that we do have to be careful of is faculty are really, really busy. And I taught one time on a quarter system, so you had three quarters, and I was on a 5-5-5 load with a total of nine new preps. So there were times that I was really struggling in running into class last minute, and then I had multiple sections and everything. And it would have been really helpful to have a 750 word that I could read in five minutes summary of the chapter, so then I could talk to the students. Because I knew the content. I just had to make sure I knew what was in there. And then for an activity sometimes drawing up an activity is not easy. If I could glance at one and get a sense of it, then I can do it. Same with the discussion questions. And so yeah, busy folks, and it’s just to help them out when they get in a bit of a jam.

John: That can be extremely helpful especially with those sorts of teaching loads, which I’ve only experienced once or twice, but it’s really challenging.

Todd: You do what you do.

Rebecca: …sounds terrifying. [LAUGHTER]

Todd: It is not easy. It’s not.

John: In the last section of the book, “A Message from Dr. Z,” you note several struggles you had in pursuing your own education. Why finish the book this way?

Todd: Well, I’m really glad you asked that. Because the last section, students oftentimes won’t read the things that are way at the end, I think the faculty are the same. And I probably have been the same too. But at the beginning of the book, I talked a little bit about some of the struggles that I had when I went to college, to almost flunking out and the fact that if one faculty member had decided to sign the form, I wouldn’t have been writing this book. And so that’s where it started. But I saved the message for the end, so there wasn’t the end of the story. And so when you go back to the “Message from Dr. Z,” that section starts with “Welcome to the end of the book,” And it’s essentially, “Let me tell you the rest of the story.” And what I did from this, there’s a great quote by E. McClellan basically says… it’s attributed to him, as there’s a lot of variations. But it boils down to everyone’s fighting a battle we know nothing about, everyone’s fighting a hard battle, it’s worded different ways. But that’s been really impactful for me, because I think if everybody knew that everybody else was fighting a battle at any given moment, then we could have a little bit more patience with individuals. But we also could say, you know, they’re getting through it, maybe I can get through it, too. So I finished the book with just a real strong narrative, in a sense that when I went through school as a first-generation college student, I almost flunked out that first semester. If you almost flunk out the first semester, just keep moving forward. And I had to work a lot of jobs, I was exhausted, but I had no money. And so I was working all the time. And so if you’re working all the time, you’re going to be tired, just keep moving forward. And I had a daughter when I was a graduate student and graduate school is already hard enough until you have a child and I almost quit graduate school because it was so hard to have a child and work in graduate school. So if that happens to you, keep going forward. And I almost ran out of money multiple times. And I would have dropped out. One time, I probably stayed in school because of $100. But I do actually know a couple of programs now, there are a couple of institutions that will give up to $500 to a student. You just show up and say, “Look, I really need $200.” And they find the students don’t abuse the system, but you don’t want someone to flunk out for $200. But in my case, at the end of this book it’s like if you’re struggling with some money, just keep that in mind. And so I just want to tell you real quickly, I don’t like to usually read these things, but just to give you the tone for the end. So I put this in there, “I leave you with the following to consider in the months ahead. Be mindful of your past, but look to the future. Listen carefully to the voices of others and find respectful ways for your own voice to be heard. Find ways to get what you worked so hard for without taking anything away from anyone else. Most importantly, always strive for more so that you have more to share. Ever forward. So that’s the tone I want to leave the students with. We’re all struggling at times and it’s not going to be easy, but if you just keep moving forward, we can make it.”

Rebecca: Speaking of moving forward. You’ve been doing a lot of writing, five books in the last five years. Are there more books coming? Are you going to take a break, like what is going on?

Todd: So I have ADHD, which means I have spent my entire life with too many things just kind of banging around in my head. So it turns out that once I started really writing, I got on a roll. I didn’t write much in my career, and it’s funny, I haven’t. And when I got rolling with some of the things, I’ve had so much fun. And so yeah, the five books in the last five years, I have another book that probably will be done in the next couple of months. And that one’s on helping new faculty to get rolling. And then I have another book that’s already signed. And that’s dealing with more with a kind of a longitudinal, how we learn and kind of walking through the learning process in a different way, which is cool. And I’ve gotta tell you, I’ve known y’all for a while there. But I’m really thinking that I need to write a book that I’m so excited about. It’s basically Dr. Z’s crazy stories, [LAUGHTER] stuff that I have kind of gone through in my life, and it’s what I’ve learned from it. So I had a student who had a grand mal seizure in my class one time, I have had all kinds of issues, lots of things have happened. And I think that there’s some stories in there that I could kind of tell because I do love telling stories. And it would help faculty, if I say, here’s how I handled this thing, and here’s what I faced. My goal would be almost the same as the end for the letters to Dr. Z for students, it’d be for faculty members of some crazy crazy stuff is gonna happen to you. And there’s ways of getting through that

Rebecca: Dr. Z’s case studies.

Todd: Yeah, that would be fun, wouldn’t it?

John: You might want to make that unreadable, though, by grad students until they’ve already started their careers, because otherwise some people might decide to back away.

Todd: No, no, no, John, they’re gonna find out that we get through with these things. And there’s also some really fun stuff that happens too, so that’s all good. You know… Alright. Maybe we don’t show it to grad students. That’s a good point. I’ve tried to defend a position. I was trying to defend an indefensible.

John: Grad students are already struggling often, so it may be best to wait until they’ve at least started.

Todd: Yeah, that’s a good point.

Rebecca: New title: Dr. Z’s survival tales. [LAUGHTER]

Todd: Oh, I’ve got to cite you on that one. [LAUGHTER] That’s good. That’s probably a better way to go with that.

Rebecca: So we always wrap up by asking: What’s next?

Todd: The book I just mentioned… what’s next. And I’m actually looking forward to getting back on the road. So kind of the what’s next is because of COVID and everything I hadn’t gone and done many workshops at campuses. And I’d love to do that. I’ve been on like 300 campuses. So it’s just fun visiting places. So I do have a couple places that I’m heading to. I was just at Anchorage, Alaska and that was really fun. And you find yourself in Anchorage, Alaska, and a couple days later, you’re in Florida. So it’s kind of an interesting thing. But I love looking at different places and traveling. So it’s been great. So the next is, I’m getting a chance to travel again.

Rebecca: So you’re getting close on your 50 states?

Todd: So, we have talked about this in the past. I’m gonna come back with my pleading of the listeners once again. I have been stuck at 49 states for about eight years. I hit my 49th state, I think it was seven or eight years ago, which I believe was Vermont… it was Vermont or New Hampshire, the order was really close. But North Dakota, it continues to be elusive.

John: North Dakota and Montana were the last two states in which an episode of our podcast were downloaded six or seven years ago when we first got started.

Todd: It’s interesting.

John: I don’t think there’s too many colleges there.

Todd: No, they’re not many colleges there. But I’m glad you did ask that. Rebecca, this is crazy. Because again, if Tim Sawyer had signed that form, I never let that go… is because anyone listening right now you never know when you’re the person who could say, “You know what, I’m going to choose something different than just letting you go.” It’s a big responsibility. But there’s times when a single sentence will change a student’s life. And so I can’t believe it when he said that, but I have now been invited to and presented in 49 states, 12 countries and four continents. Just amazing. And I would feel better if North Dakota would just call me, that would be so nice.

John: The last continent might be tough.

Todd: Yeah, there’s one continent that’s really tricky to get a gig in.

Rebecca: It could happen.

Todd: It’d be helpful if there are people who live there. [LAUGHTER]

John: The penguins are not that impressed.

Rebecca: You could just invite yourself. [LAUGHTER]

Todd: See that was the rule, by the way.

Rebecca: Oh, that’s right. That’s right.

Todd: Yeah, the rule was you had to be invited, so you can’t just show up someplace and start talking.

Rebecca: Well, we look forward to keeping tabs on your 49 states. Next time we talk to you. It’s always a pleasure, Todd.

Todd: Thank you… appreciate the opportunity to come in and I’llI get the 50th state,I will give you a call and maybe we can do a show about my 50th state. [LAUGHTER]

John: And I think sometime before that might be nice as well.

Todd: Wow. There’s a little pessimism for you.

Rebecca: Geez. [LAUGHTER]

John: Well, that may take a year or two.

Todd: Gosh John… Rebecca. Wooh. That’s tough. That’s brutal. Alright.

Rebecca: I was thinking that was gonna come soon, Todd. Sorry.

Todd: This is a mic drop time. Why don’t you go ahead and do the outro at this point. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

265. The New College Classroom

Despite all that we have learned from cognitive science about how people learn, the most common form of classroom instruction still involves students passively listening to a lecturer standing at a podium at the front of the room. In this episode, Cathy Davidson and Christina Katopodis join us to discuss alternative approaches that treat student diversity as an asset and allow all students to be actively engaged in their own learning.

Cathy is a Distinguished Professor at the CUNY Graduate Center, the author of more than twenty books, and a regular contributor to the Washington Post and the Chronicle of Higher Education. She has served on the National Council of Humanities and delivered a keynote address at the Nobel Forum on the Future of Education. Christina is a Postdoctoral Research Associate in the Transformative Learning in the Humanities Initiative at CUNY and has authored over a dozen articles on innovative pedagogy, innovative pedagogy, environmental studies, and Early American Literature.  She has received the Dewey Digital Teaching Award and the Diana Colbert Initiative Teaching Prize.

Show Notes

Transcript

John: Despite all that we have learned from cognitive science about how people learn, the most common form of classroom instruction still involves students passively listening to a lecturer standing at a podium at the front of the room. In this episode, we explore alternative approaches that treat student diversity as an asset and allow all students to be actively engaged in their own learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Cathy Davidson and Christina Katopodis. Cathy is a Distinguished Professor at the CUNY Graduate Center, the author of more than twenty books, and a regular contributor to the Washington Post and the Chronicle of Higher Education. She has served on the National Council of Humanities and delivered a keynote address at the Nobel Forum on the Future of Education. Christina is a Postdoctoral Research Associate in the Transformative Learning in the Humanities Initiative at CUNY and has authored over a dozen articles on innovative pedagogy, innovative pedagogy, environmental studies, and Early American Literature. She has received the Dewey Digital Teaching Award and the Diana Colbert Initiative Teaching Prize. Welcome, Cathy and Christina.

Cathy: Great to be here. Thank you for having us.

Christina: Thank you so much.

Rebecca: Today’s teas are:… Cathy, are you drinking tea?

Cathy: I am drinking English breakfast tea.

Rebecca: Yes. [LAUGHTER] And Christina?

Christina: I am drinking English breakfast tea.

John: And I am drinking wild blueberry black tea.

Rebecca: I forgot what kind of tea I made this morning. [LAUGHTER] I have no idea. I made nice loose leaf tea this morning, and it’s tasty, but I don’t remember what it is.

John: So you’re drinking a tasty tea.

Rebecca: I’m drinking a tasty tea this morning.

Cathy: Will I be kicked off the show if I say I’m drinking coffee?

John: About a third of our guests do. Yeah, and sometimes water.

Christina: I also have a Diet Coke. [LAUGHTER]

Rebecca: Sneaky, very sneaky.

John: And I had that right before I came over here.

Christina: We’re drinking all of the things.

John: We’ve invited you here today to discuss The New College Classroom. Could you tell us a little bit about how this book project came about?

Cathy: I can begin. This is actually the third in a series I called the “how we know” trilogy, which I began in 2011 after I stepped down as Vice Provost for Interdisciplinary Studies at Duke University, my previous employer, and was really interested in the science of attention and the neuroscience of learning. So that book really is about the neuroscience of learning. Then I wrote a second book called The New Education which came out in 2017, and was re-issued in a post pandemic version just this past spring. And it’s called The New Education. It’s really the history of higher education and why we inherited the forms we have now and how much higher education was re-created for and rebuilt, redesigned explicitly for the industrial age. And then I wanted to do a kind of installation guide that actually showed people how we do these things, how you take our knowledge of how the brain works, take our knowledge of learning, and how you take our knowledge of history and why we’ve inherited this very cumbersome history and actually do something new. And I thought it would be ridiculous for someone at the end of a long career to be telling other people how to teach. And I had the great fortune to be working with this… Christina will close her ears right now, she gets very embarrassed when I say this… but this utterly brilliant scholar, an Americanist environmental scholar, sound studies scholar who also had written several essays on pedagogy and had won all the teaching awards. And I asked her if she’d be interested in co-writing a book with me. And we started writing it when I was a senior fellow to the Mellon Foundation in my beautiful office overlooking the courtyard of the Mellon Foundation. And then the pandemic hit. We made a pledge to one another that we’d meet every Tuesday and Thursday and write together and we literally wrote, rewrote, re-re-re-wrote, and then re-re-re-re-re-wrote [LAUGHTER] every word together during the pandemic. So when people say, “Can you have a real relationship? Can you have a real project during the pandemic?,” we would say “Absolutely.” And that’s actually kind of key to the book, ‘cause we talked about learning in all its facets online and face-to-face. But that’s the very long version of how I’m the luckiest author in the world to have been able to work with Christina.

Christina: I’m the luckiest. [LAUGHTER]

Rebecca: It must have been very nice to have the stability of consistent writing times with each other during a time that was so unstable.

Christina: Yeah, it was a lifeline. I live in Brooklyn, near the Barclays Center where a lot of the Black Lives Matter protesting was happening here. And kids will be playing in the street. They had to close the streets on the weekends and weekdays so that kids could get time outside and neighbors could gather together outside where it would be safe. And kids playing on the street, drawing all these beautiful things in sidewalk chalk, and then you’d see police completely decked out, batons out, pepper spray, all kinds of things, getting ready for the protest that was just completely peaceful protesting and a police force that was just prepared for something wildly different. And it was a really dark time in trying to protest and think of a better way, a more equitable institution, an institution that could prepare students for the world, to be citizens of the world and also to fight for social justice and racial justice, and getting together and writing this book. It felt we have even more purpose more so than ever, in writing this and leaning into the active learning methods that really prepare students to participate and engage with the world.

John: You begin the book with something which actually serves as a very nice introduction to much of the rest of the book, which is a story about a 50-person department meeting in which no one was responding to the department chair. Could you just share a little bit about this anecdote?

Christina: Sure. So I was in that department meeting, it was a meeting in which we were given a really big task of imagining the goals for the department for the next 10 years. This is a review that happens every 10 years. And the department chair was standing at the front of the room behind the podium and was like, “Alright, so what are our goals for the next 10 years?” [LAUGHTER] …and everyone was quiet. Because as you can imagine, everyone has things that they want to change, things that they think could be better, and no one wanted to be the first to speak. And it really just felt like one of those situations where there’s a bunch of dry straw, and someone could just light a match. And so I was like, “Okay,” I knew everyone there, and I was familiar with the chair. The chair was very supportive of students, and I knew he was willing to listen. And then I said, “Hey, can we just talk to someone next to us and come up with a few ideas first, before we speak with the whole group?” He’s like, “Sure.” And this is what’s called think-pair-share where everyone thinks about a question and then they pair up with someone next to them or a small group of people. And then when we’re all done, we come back and share what we came up with. And five minutes go by and he is trying to get the attention of everyone in the room, the room has exploded in all this conversation, but everyone is smiling and enjoying talking to the person next to them. They’re thinking more hopeful thoughts with their generous colleagues and students and faculty all together. And when the chair was finally able to call everyone back to order… I’ve done this in 8 ams… the poor people who teach next to me have said that my classes are too loud at 8 am. [LAUGHTER] And Cathy has done this with a lot of people every time she gives a talk, and it takes a long time to get everyone to come back to order. And then people were so eager to share what they had talked about in their groups. And there was a little bit of anonymity, because it was like, “Okay, everyone in my group said this, not just me,” [LAUGHTER] and everyone was willing to share and we started to envision some really beautiful goals for the next 10 years that were really hopeful, that were imaginative, and creative, and beautiful, rather than starting with critique. Or sometimes what happens is, if one person says something, then everyone else kind of jumps on that train, and this way, diverse number of ideas coming out of these different separate conversations. So that’s why we do it.

Cathy: If you want an education-ese term this is called an inventory method. It’s the opposite of the standard seminar where you ask a question and those same three students raise their hand dutifully and answer the question. And sociologists of education have studied who those students are who raise their hand. And they tend to be a good match for the professor in class and race and gender and family background, family income… the people who are most into the class, most likely to get an A plus, most likely to go on to graduate school, most likely to be professors, replicate their professor. And that’s one reason why only 1% of Americans have a PhD and 25% of the professoriate has a parent that has a PhD. We have a system that’s a closed system. When you do an inventory method, like the one that Christina uses, think-pair-share, everybody in the classroom contributes. In that case, it was a meeting. But that’s true in a classroom too. Sociologists of education also tell us that 20% of students graduate from college without ever having spoken in a class unless they were required to speak by a professor. But if we really believe that higher education is about empowering students, not just giving them content, but giving them the tools to be experts themselves, then they have to learn how to articulate those ideas. And of course, some people are shy and then having them write on an index card means they’re still participating , even if they’re too shy to actually say something in class. An incredible method that I learned from a second grade teacher and I’ve done in many situations. Christina was alluding to the famous one I tell all the time about trying think-pair-share with 6000 International Baccalaureate teachers in the Philadelphia Seventy-Sixers auditorium. So they’re all thinking and pairing and sharing on these jumbotrons in the auditorium.[LAUGHTER] It was great. And I’ve also done it with the top 100 performing CEOs of the Cisco foundation. I did this with the Board of Trustees at Duke and John Chambers, the CEO of Cisco was on, he said, “I need you to talk to my executives, because there’s nobody more likely to be reticent about giving comments than one of my CEOs who’s talking to me. Power does that. There’s a kind of silence that happens in the face of power, whether it’s with second graders or a department meeting like Christina was in or students every day in our classes.

Rebecca: I think what’s really interesting about both of the stories that you’re sharing is that we’ve been in those situations where the silence is overwhelming, we’ve maybe have been the person running the meeting or the classroom or also been the person in the audience.

Cathy: And it feels awful, it feels terrible. My students call it’s playing silence chicken. [LAUGHTER] That’s a great term.

Rebecca: It’s a perfect descriptor for sure. So given that we’ve all had these experiences, and also have experienced the opposite. I think probably all of us, or at least most of us have experienced those engaging opportunities. Why do we always default to the one that doesn’t work? [LAUGHTER]

Cathy: Every structure we have in academe tells us our job is to learn from the master. And I use that word in quotation marks, but pointedly, and then repeat back what you learn on a final exam and that’s how you get As. And the students in our classrooms, they’re the winners, not the losers. They got to college because they learned that lesson, Freire calls it the “banking model,” where it’s my head dumping and depositing stuff into your head. We also have studies that go back to the 1880s, not 1980s, 1880s, the Ebbinghaus experiments with memory that tell us we forget 75% of what we’ve learned for that exam within weeks after the exam is over. So it’s not an effective way to learn, but it’s the way we learn. So we’re being reinforcing in a way of learning we only use in higher education and formal education, we don’t learn new skills that way, when it’s not commonsensical to learn from a lecture. I don’t learn how to play tennis from a video. I might look at a video, but that’s not how I get better. I practice and I improve and someone corrects me, and then I change what I’m doing. We all know that. But, for education, we’re told, learn from the sage on the stage. That’s what you’re supposed to do. That’s how you excel and our students have learned how to excel. It makes complete sense that they would think they learned more from a lecture than when they contribute. They’re deferring to the authority in the room.

Christina: And I think that this has been ingrained in them for a long time. We’ve been ranked and rated since birth, literally the first minute of birth ranked against others. And this one wonderful researcher, Susan Engel. She’s at Williams College, developmental psychologist, and she cites a study where kids on average before they go to school, when they’re at home asked about 27 questions per hour. And anyone who has a toddler, sometimes it is well more than 27 questions per hour. [LAUGHTER] My toddler just started asking why about literally absolutely everything. He’s about three. So right on target for asking lots of questions, for being naturally curious about the world. She talks about this. It’s called epistemic curiosity. And Dr. Engel says that once kids go to school, they ask on average three questions per hour, which is just a precipitous drop in getting to ask your questions, getting to know the world around you through engaging with it. And so by the time they get to college, they expect to learn more from a lecture, that they are not experts in the room, and that the person at the front of the room has all of the answers. And they have been trained through standardized testing to believe that there is a right answer, and they just need to be able to get to that right answer. That doesn’t train them to take on the world’s toughest problems and be problem solvers, to find alternative solutions, to think outside the box. And maybe those problems in the world are caused by believing there is a right answer and not thinking that maybe it’s me sitting in the room with all of these other people sitting in the room who could find a better solution or a better answer than the one that that one person has. And that’s how we change society. That’s how we transform our institutions. And I think they’ve just been ingrained in this system for so long, that by flipping it around and saying no, there are 50 different ways to answer this one question or 50 different ways to solve this one mathematical equation, then it’s more up to them. It’s giving them a little more responsibility and autonomy so that they can practice using it when they get out into the world rather than thinking, “Oh, there’s someone smarter than me who clearly has that figured out and they’ll take care of that problem.” That doesn’t really give them the kind of responsibility and accountability that I think that we all need in the world.

Rebecca: What you were talking Christina, I was thinking about a conversation I had with my kindergartner over the weekend, who does still ask many, many questions, some of which I do not know the answer to and so I don’t remember what she was asking me but whatever it was, I did not know the answer. And I said, “I don’t know, we’ll have to look that up” and she’s like, “I’ll ask my teacher, she knows the answer.” She’s in kindergarten, she’s only been in kindergarten for two months.

Cathy: I interviewed kindergarten teachers and first-grade teachers about learning. And almost every kindergarten teacher said, “when kids come to them, they think they can do all these things like ‘I love math, I love music, I love art, I’m an artist,’ by first grade, within six months into the first grade, they know, ‘I’m not good at art, I’m not good at math, I’m okay at language, but I’m very poor at…’ and already have absorbed those kinds of lessons about themselves.” And what you said is great, because it’s like the authority of the teacher is already happening. And then what happens when you absorb those lessons into a self definition. And that’s what we’re working against with active learning, is not just definition of the teacher, but definition of yourself in your role as a person.

John: And that’s hard to correct at the college level, because students have been indoctrinated in this from their very first exposure to educational systems. And when faculty do try using active learning, they often get a lot of pushback from the students. And that’s a challenge for new faculty where their teaching evaluations may have some impact on their continued employment. So it’s a difficult cycle to break.

Cathy: Yes, and it’s one reason why we include the research, serious research with any thing we offer to faculty members about what they can do in their class. And we also talk about the 2014 meta study in the publication of the National Academy of Science that looked at every possible way of evaluating learning and said if this had been a pharmaceutical follow-up study that Eric Mazur, who’s one of the inventors of the flipped classroom, did at MIT because his brilliant MIT students, were all sure they were being shortchanged by active learning. So he had them read serious scientific studies of active learning and then they all thought they were doing great by having active learning, and they thought they were better. So it’s about using the methods that speak to people in order to change the methods because unless you address the actual present situation of the audience, of the students, of the people you’re addressing, you can’t change things, you have to honor that present situation before you can move to something else and make a structural change beyond that.

Rebecca: A lot of the current system of higher education, as you mentioned, is based on a really different era, a really different audience of students and our student populations have changed, become more diverse, there’s more people going to college now than before. So how do we help students who have been through this system that has not really invited them to the table to really get involved. We share some of the research, and what are some other ways we can support them on this endeavor, and to continue helping us change the system.

Christina: Some really great pedagogy out there. One I’m thinking is an assignment that Erin Glass does with her students where they read terms and conditions for all of the technology that they’re using on their campuses. And they have to closely read them and critique them to help them become more aware of capitalist surveillance and what they are required to sign up for. And then they write a critique of that, of the university and of the system that they’re being signed up for, so they’re not only learning more, they’re learning digital literacy. They’re also learning more about the institution that is guiding these things that they’re signing up for. And they’re becoming better critical readers in general. And they’re also talking back to that institution and saying you could be doing better. And so there are ways in which we can give our students real-world problems that are immediately close to them and to their experiences of education, and task them with coming up with something better. And it’s really wonderful that students who come from all different kinds of educational systems to get together and think of what could be done better. Or Cathy also does this where at the end of the semester, a final project could be like, come up with a better syllabus. I don’t know if you want to speak to that Cathy, but I love this assignment.

Cathy: Yeah, I love to end the class where I be in the next class, like I say, “Okay, we’ve had about 12 weeks or 16, depending on the institution that I’m at, together and we’ve done this as a syllabus, and you’ve contributed to the syllabus, the last assignment, and sometimes we make this even as a final exam, is to make a syllabus the next people who take this class will inherit. Put your stamp on it. What did you like? What didn’t you like? That’s an incredible activity and it means it’s but again, using an education-ese term, it’s metacognition, too, because it means students are looking back over everything they’ve learned, which is the best way to beat the Ebbinghaus 75% forgetting because you’re actually processing it, analyzing it, and then trying to come up with some new version that you’re bequeathing to somebody else. It’s a legacy that you’re bequeathing to somebody else. It’s a marvelous exercise and you can do it in any kind of class. It’s quite fascinating to see the different ways people can come up with things. So often people say, “Well, you’re an English teacher, so of course, you’re flexible.” But what about… there’s somebody named Howie Hua, who’s a professor at Cal State Fresno, who on Twitter almost every day he comes up with what he calls mental math problems. And he’ll ask something seemingly simple that you think there’s no other way to solve that, like, “Add in your head 24 plus 36? How did you do it?” And dozens of people respond with different ways to add whatever those two numbers are that I just said, in their head. And it’s fascinating, because the point he’s making is it’s not about the right answer. It’s about, not only understanding the processes, but understanding all the different tools that we can have in order to be better mathematicians, more passionate about science or more passionate about all kinds of learning. So I think any assignment that gives students the tools and allows them, and even better is when they can pass on those tools to somebody else. When Erin Glass does that assignment, they not only critique, but they come up with their own terms of service. What are the terms of service for our class? What’s the community constitution for our class? What are the community rules we’re going to form in this class? …and so they’re already invested in a new kind of structure even before you start populating what that structure is going to look like.

Christina: And I think from the first day, you can really set students up for something different. One thing that Bettina Love does in her classes is she has everything set up and she goes, “Okay, now you tell me when the deadlines are going to be for this.” And so students look at their calendars, look at their schedules, and determine when things are going to be due so that they’re not all due in the same week as midterms or as in the same week as finals. That’s a very student-centered approach, asking students to come up with the learning outcomes of a class. And even in the most restrictive situations, you can add learning goals and learning outcomes, even if yours are set by the department. So I think also asking students what needs to change to better serve them and centering them in that conversation, you can start with your class and then start to think more broadly about a department or a whole institution.

Cathy: What I love about all of these different things, and we’ve borrowed them from other people, one of the things we did was we interviewed so many people, academic twitter was very helpful for that, to find out what they were doing and profile people and amplify people who are doing amazing things. What works best is when something specific, if you say how should we change this class, you get silence chicken. If you say here are the 10 learning outcomes that our department requires us to write, do you have anything else you’d like to add? Maybe work with a partner and come up with your own additions to these required learning outcomes? Students come up with beautiful, soaring, inspirational things that just make you aware that if they’re allowed not to be cynical, students want to have agency and want to have something that will help them in the rest of their life, it’s pretty scary to think about the world out there. And they want and need these things and want to be participants in the shaping of their own life and in their own agency.

Christina: And you’re reminding me too, that there’s this widespread movement right now to rename office hours, to call them student hours, because students are so used to going to the office being a punitive experience. And student hours really welcomes them to come with maybe more than just “I’m having difficulty with this problem set.” “But I want to talk about my career, what can I do with this degree? Or I’m having difficulty with x? Can you help me with y?” I think it really better serves them as well to rename it student hours to show that we actually really want you to come and we really want to talk to you. I don’t want to just sit alone in my office. [LAUGHTER] I want you to be here.

John: What are some other activities or ways in which we could give students a bit more agency? You mentioned the reflection on the syllabus and rewriting a new syllabus at the end. And you also hinted at having students be engaged in the syllabus itself. What are some other ways we can do this during the course of the class?

Cathy: I often quote our friend Jonathan Sterne, who teaches at McGill University because this is the most counterintuitive one. Jonathan teaches various versions of mediasStudies, media information, technology, and disability studies. He himself had throat cancer, and he talks through a voice box that he himself helped to design… a remarkable human being. He teaches classes of 400 to 600 students, and people say there’s no way… how in the world could that be active learning? He hands out index cards and at the end of every class students write down an answer to something he asks. He might ask “What did we talk about in class today that you’re still going to be thinking about before you go to sleep at night? And if there was nothing, what should we have been talking about?” Today’s media if you don’t have anything to say about media in the modern world, something’s misfiring. He has a blog about his learning, and he’s charted how well students have done since you’ve done these simple exercises of having students report back after every class. He also works with TAs who have special sections, and they take the 15 cards from their section, so they know what questions the students have before they go into the session. And then he uses some of those to spur his next assignment. He does this kind of cool thing where he spreads it all the cards and says, “Well, John said so and so. And Deborah said so and so. And Rebecca said, so and so. And Christina said so and so.” So he makes it interactive. He also does an incredible thing. He says, “With this many students, I have to use multiple choice testing, and I know how impoverished multiple choice testing is.” So he sets his students a creative assignment each time, sometimes it’ll be a piece of notebook paper, sometimes it’ll be an index card, and he’ll say, “You can write any crib sheet you want, go back, and you can do anything you want from the semester to help you do well on the multiple choice exam.” And what he knows is what they write on that crib sheet is the learning, right? It’s not filling in the ABCDEs, it’s the learning he also leaves some portion, it might be 5%, might be five points, it might be10 points, depending on the system he’s using. He’s Canadian university, so it’s a slightly different system than in the US. But he leaves some amount free, and students hand in both their multiple choice exam and their crib sheet. And he gives extra points for the crib sheets. He’s even done art installations with some of the crib sheets that students have done. But the point is the way you review a class and organize the knowledge onto some very prescriptive sheet, and he says the prescription is extremely important… he’s a composer as well. So he knows how it’s important is to have rules and to play with those rules. And it’s almost like a game. That’s where the learning is happening. So even in the most restrictive situation, you can still do active learning and learning that’s meaningful to how your students learn and how they retain and how they can apply that learning later.

Christina: A colleague of mine, Siqi Tu, also has her sociology students come up with some of the questions that will be included on the final exam. And you can do this with a multiple choice or a long answer type of exam, and the students develop a question. And then when they submit their questions to the professor to review, they also need to include what skills are we assessing with this question? And is this the right answer? What are the various ways you could get to the right answer? And if the right answer is D, then why are you offering A, B, and C as other but wrong answers? Why are they not the right answer? …things like that, to get students to have this kind of command over what they’re learning, why it’s important, what is worthy of being assessed, and how to go about assessing it and testing that knowledge. They have a lot more agency then in how they are being evaluated. And so their expertise is also being solicited there. And the majority of the learning is happening in creating the question and explaining all of the pieces of how that question has been crafted and what the right answer is and why. And then, at the end, she includes at least a portion of the student-generated questions on the final exam. And so it not only gives them great exam prep, but then they also know better what to expect. And they have more agency and control over how they’re being evaluated.

Rebecca: I really enjoyed some of the titles of the chapters in your book, for example, “group work without the groans.” [LAUGHTER] And I thought, yes, group work without the growns. So they resonate.

Cathy: I used to consult quite a lot with business. And just because of that I helped create the Duke Corporate Education Program, which was for returning executives, when I taught at my previous institution. And the number one expense of management experts who are brought into corporations to help manage more effectively is to help them with group work. And in classrooms, sometimes even in sixth grade, we say “Work in your groups and then do something amazing.” Well, no, we know what’s going to happen. That one person who raises their hand, they were one of the three who raises their hand, and in the group, they’re the one that does all the work, then there’s somebody else who kind of goofs off, and then there’s somebody else who does nothing at all. And we know those patterns, and it’s horrible for all three, it’s horrible for the person who always steps up, they’re not being pushed, and it’s horrible for the person who does nothing or the person who goofs off. So when I do group work, I have students write job descriptions. They write out job descriptions for who is going to do what in the group and I also love to have them do an exercise that I call superpowers. What’s your hidden superpower? What’s your three things you do that you think have no relevance at all to this group but that you know you do really well? It might be playing video games. I’ve had people say they were a clown. I found out that my executive director of the program I run was a professional clown. I didn’t know that. She’s a gorgeous young woman and she was a clown. She’s the last person I would have thought of who would be a professional clown. People have these skills, and what they find out in a group is those skills are not irrelevant, because you’re talking in group work not only about coming up with a product, but about interrelationships. And how you can make all the different parts of your personalities work together coherently to create a final product. Also, I have students put that on their resume. I have them look online at what employers most prize. And it turns out collaboration… duh…because that’s what they spend their money improving. Somebody who can be a great collaborator is somebody you want to hire in a job. So my students can not only put that on their resume, but have a wonderful example. You don’t have to say it’s in a classroom. I worked on a project with four others, and we took that project in from idea to implementation, and my role was the firestarter. That’s a term from computer scientists. I’m especially good at coming up with new ideas and presenting those ideas to a group. Fantastic. And then they don’t grown, they realize they’re learning a skill, not being put back into a pattern that they themselves hate and are embarrassed by or resentful of.

Christina: And I think in addition to telling them why group work is important, that they’re going to end up working in groups for the rest of their lives. Everything I’ve done in any job has been collaborative to some degree, and mostly a lot, like really collaborative. And so I just kind of tell them, “This is not busy work, this is actually good practice for the rest of your life, because at the end of the semester, you’re not going to work with these people anymore. But if you’re in a job, the only way to leave that group is to leave that job and the stakes are so much higher. So first of all, this is good practice.” But I think also, we sometimes neglect to offer students the structures that they need to feel confident in their grade for group work, and to feel confident going into group work. So from the get go, giving students structure for the group work, like a checklist of jobs or asking them to come up with a checklist of tasks that they need to complete, assigning roles, like putting a name next to each item on that checklist, so that it’s clear who is doing what. Teaching them a skill like that is teaching them how to delegate authority, how to be a good entrepreneur, a leader, and pointing that out to them that by creating this checklist and putting everyone’s name next to everything, you’re delegating authority, you’re learning these leadership skills that you need in the workforce, and helping them to understand that it is absolutely okay and totally normal to feel social anxiety before going into a group, particularly after a pandemic, when we really lost the ability to make small talk.

Cathy: It’s exhausting. [LAUGHTER]

Christina: It’s exhausting. It’s exhausting. It is we lost that skill. Oh my goodness,

Cathy: Today happens to be a Monday and I had brunches on Saturday and Sunday and took a nap after each one. I can’t remember how to do brunch anymore. [LAUGHTER] And we’re all in that situation.

Christina: It’s true. I just wanted to add one more thing, which is that it’s not just social anxiety, but it’s also anxiety about grades and grading. And we have a whole chapter about grades. Because no one likes grading. No one likes grading. And I think it’s important too for students to know that they’re being graded and assessed fairly on group work. I’m so having that checklist of roles and turning that in at the end to show who did what is really important. And inevitably, it’s funny because we also put faculty into groups …that transformative learning the humanities… where I work, and this inevitably happens with faculty too, that they get really anxious about working when the ideas aren’t gelling together. And I think it’s really important for students to know how they’re being graded. And if someone’s ghosting or someone’s not pulling their weight or not showing up, give everyone else extra credit for helping to make that work, helping to reach out to that person who’s not showing up or ghosting: “Are you okay?” And a lot of times that person is not okay. And they needed someone to reach out and ask if they’re okay, they need that support. So offering the students who unclog a problem, you can give them a plunger award, literally, that happens in a lot of groups, or do something to recognize someone who goes above and beyond to help resolve those kinds of conflicts and issues, rather than it feeling like “Oh, but my grade is being hurt by someone else.” I think it really helps to foster collaborative community and a learning community where everyone is important. Everyone is valued and everyone needs to be okay for the group work to be successful. How can we help our colleagues, our peers.

Cathy: Around 2005 to 2010 years for my organization HASTAC which I co-founded in 2002… NSF now called it the world’s first and oldest academic social network… we created a wiki and we went to this guy in a garage and asked him if he would help us create a wiki and then the next year he launched Wikipedia… that was Jimmy Wales. [LAUGHTER] So that’s how old, that’s how ancient this is. But with that world, we were dealing a lot with open-source computer programmers. When they would do a job, when a job would be posted on Stack Exchange or another open source site, they would have to find a partner that they’d never worked with before and know if that person was reliable, somebody who could complement their skills, and so they do a badging system. And I worked with the Mozilla Foundation on creating badging systems, where you would write down all the criteria that you need to accomplish a job and you never gave a negative, you never give a negative, you just give somebody a badge if they did something great in doing that, and then somebody else who comes by and wants to look to see if they want to work with that person sees where they’ve been given the right badges and says, “ooh, yeah, those are the things I do poorly and those are things what you do well.” Again, HASTAC calls that collaboration by difference. Not everybody has to do everything perfectly, but you have to know, you have to have an inventory of what people contribute. So in my classes, also, when I’m doing group work, I’ll have students not only write their own job descriptions, but write a list of the qualities they think are most important for the success of the group as a whole. And after every week, when they come together, I’ll have them give badges to the people. I don’t even say how many, what percentage, just give a badge to someone who you think really showed up this week, and give a badge in a different category. You don’t have to tell the person who never gets a badge from any of their peers that they’re not pulling their weight. And then that can be a first step, as Christina said, to doing something like reaching out and saying, “You must feel terrible that nobody’s given you a badge in anything. What’s going on? Is it indifference? Is something going on your life? What’s happening?” Yeah, and it doesn’t have to be punitive. But when you see that none of your peers are rewarding you, sometimes it’s like, “Darn them. I’m doing all the work. But I’m a shy person. So I just make the corrections.” That often happens among computer programmers who often are not the most voluble, personable, people. They used to wording, we’re doing code? And someone will say, :”No, no, no, I was fixing all the code you didn’t see, you just didn’t look and see that I was fixing all the code.” And then you can make an adjustment to and it’s an incredibly important adjustment. I’m working with somebody now who was Phi Beta Kappa, three majors, straight As, et cetera, et cetera, and never spok in college. And when she hears these methods, she says, “This would have changed my whole way of being in college.” Instead of feeling shame of all the people who should not feel shame, she felt ashamed that she wasn’t contributing. And because she was never offered an opportunity to until she wrote this brilliant final exam or brilliant final research paper when her teachers knew but in class, she felt like she was failing. That’s horrible. That somebody that brilliant would ever feel like they weren’t doing a great job.

Christina: I think it also asked how to get students excited. And thinking about what Cathy just said, I like to frame group work as an opportunity to practice being a step-up or a step-back person. So if you’re normally a step-up person, like everyone loves that you’re the first to volunteer to help. It’s great to be a step-up person, but sometimes that doesn’t leave room for stepping back and taking into account all of the things that have been said, and reflecting on the larger picture and finding the forest through the trees. And so I invite students to try to practice if you’re generally a step-up person, try being a step-back person. And if you’re generally a step-back person, try being a step-up person and see how it goes. And I also put all of the really loud step-up people in a group together, and I put all of the really quiet step-back people in a group together, because at some point, they’re going to have to talk and the step-up people need a way to regulate who is talking. And so I think a lot of us try to distribute those people through groups, and that can really change group dynamics. So I like having them all together in various ways to feel comfortable being among peers and navigating those roles and being more aware, calling to mind those roles before group work, so they could get excited about trying something new, and recognize that this is always just practice.

John: Will these methods help to create a more inclusive classroom environment?

Christina: These methods are inclusive because they solicit participation from every single person in the room. Inventory methods achieve total participation, that’s a term from the American Psychological Association, 100% participation, not just the hand-raising few. So these methods are inclusive.

John: There has been a lot of criticism recently concerning the way in which traditional grading systems cause students to focus on trying to achieve the highest grades rather than on learning. Do you have any suggestions on how we can focus student effort on learning rather than achieving higher grades?

Christina: Shifting the focus from grades to learning? A very quick answer would be thinking along the lines of Carol Dweck and using a growth mindset… that we are all learning… that you can do more assignments that are completion-based or labor-based to a satisfactory degree rather than A, B, C, D, F. And I think that one really great model is from Debbie Gail Mitchell, who teaches chemistry in Denver. She decided that an 80 is achieving proficiency. And the goal is to achieve proficiency. And so if you get an 80 on an exam or an assignment, then you receive the total number of points for that exam or assignment. And there’s a total number of points to achieve for the whole class. And so assignment or exam adds up to that. And so students stop grade grubbing or worrying if they’re getting an 83 or an 84. And they focus more on learning.

John: We always end with the question: “What’s next?”

Cathy: At one of those brunches This weekend, I had the privilege to meet someone named Matt Salesses, S-A-L-E-S-S-E-S, who is a novelist who just took a job in the MFA program at Columbia, who’s also written a book called Craft. And the book looks at how we teach writing in writing workshops by looking at novels and noticing how often white-authored novels don’t tell you the race of the white characters, but do tell you the race of the non-white characters, and how much craft through all fields as well as in writing workshops often assumes a putative expert in a putative subject as being white. And then everybody else gets defined. So even when we do the terms like diversity and inclusion, there’s an implicit grounding that the person who has craft and earns that craft is going to be from the dominant race. And it’s a really interesting book that looks at the Iowa Writers Workshop and the principles by which it was written up. You could have done craft as a way that higher education was set up. So many of the people who started and set up the metrics for higher education in the late 19th century were in fact, eugenicists, they really believed there was a biological reason for racial superiority and created a system that reified that prejudice that they had. So that’s just a parting. I just happened to be at a brunch at one of those exotic face-to-face human real experiences this weekend where I met an astonishing person with an astonishing book that I’m thinking about more and more. So that’s a what next for me as of yesterday, but that’s the wonderful thing about active learning is yesterday always has something interesting you can learn from.

Christina: My what next is I’m working on an article right now with a colleague, Josefine Ziebell. We’re both sound studies scholars, and also interested in pedagogy. And we’re looking at the school-to-prison pipeline, and the ways in which school soundscape mirrors the carceral soundscape, and how to give students more sonic agency. And so thinking about silence in the room, thinking about voice, and in what ways speaking up can challenge authority and how that can cost you your life in the worst scenarios. And also just like bells ringing, the time ticking ways in which school soundscape can contribute to the school-to-prison pipeline by regimenting everything about a person’s body and depriving them of their sonic agency, where making noise is considered inappropriate when that is exactly what we need to do to transform these institutions. And so I guess that just studying that I find really interesting right now with Josefine.

Rebecca: Well, thank you so much for such fascinating conversation today and things to think about and wonderful teasers for your book.

Cathy: Thank you very much.

Christina: Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

248. Reframing Academic Expertise

Professors are generally represented in popular culture as white male experts who dispense knowledge to their students through lectures. Young female professors are often encouraged to portray themselves as authoritative figures, even when this role does not reflect their personalities and their educational philosophies. In this episode, Rebecca Scott joins us to discuss how she has rejected this stereotype by sharing vulnerability and building classes that rely on the co-creation of knowledge.

Show Notes

Transcript

John: Professors are generally represented in popular culture as white male experts who dispense knowledge to their students through lectures. Young female professors are often encouraged to portray themselves as authoritative figures, even when this role does not reflect their personalities and their educational philosophies. In this episode, we discuss how one professor has rejected this stereotype by sharing vulnerability and building classes that rely on the co-creation of knowledge.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Rebecca Scott. Rebecca is an Assistant Professor of Philosophy at Harper College, and also a guitarist and vocalist in the band Panda Riot, which just released their fourth album. She’s also the author of a chapter in Picture a Professor, edited by our friend Jessamyn Neuhaus from SUNY Plattsburgh. Welcome, Rebecca.

Rebecca S.: Thank you. I’m excited to be here.

Rebecca: Today’s teas are… Rebecca, are you drinking tea?

Rebecca S.: I’m drinking coffee.

Rebecca: Oh, a coffee drinker….

John: Well, we drink coffee too, once in a while.

Rebecca: Very occasionally, but not on this podcast. [LAUGHTER]

John: I am drinking Irish Breakfast tea this morning.

Rebecca: Oh, it sounds like a good theme. John. I have English breakfast this morning.

John: We’ve invited you here to discuss your chapter on “Reframing Academic Expertise through Vulnerability and Metacognition” in Picture a Professor and some of the other work that you’re doing. But before we discuss this, could you talk a little bit about your institution, and the courses that you teach?

Rebecca S.: Harper College is a two-year college, or community college, in the suburbs of Chicago. And, because we’re a community college all my classes really are introductory classes or don’t have any prerequisites anyway. So I mostly teach Intro to Philosophy, Critical Thinking, and Ethics, then I occasionally get to teach Biomedical Ethics and the occasional honors class. And in the fall, we have a new program that’s a social justice studies distinction. So I’m going to be teaching an Intro to Philosophy class that’s specifically for that Social Justice Studies program at our college.

Rebecca: Your chapter title is really intriguing in Picture a Professor. Can you talk a little bit about or give us a little teaser on how you reframe academic expertise through vulnerability and metacognition?

Rebecca S.: In that chapter, when I was approaching this question of how do we address these kinds of biases against professors who come from marginalized identities, the way that I came to this question is, when I was first starting out, people would give me a lot of advice about, “Oh, you know, you’re young, and you look young, and you’re a woman, and you have to be really careful.” And I feel like a lot of the advice I got was that I needed to be like, strict and I needed to be like, hard, and I needed to take on this authoritative stance so that people would give me credibility. And it just felt really not who I was. It just wasn’t me. And it works for some people, I think, and that’s great. I’m not in any way saying that people shouldn’t necessarily do those things. But for me, it really just was not the way that I wanted to teach. And so I’m always trying to figure out how can I have authority and credibility in a way that feels authentic to who I am. And so this got me thinking about the ways that part of the problem with the sort of stereotypical image of the professor is not necessarily that we don’t have enough different kinds of people occupying the role of the professor, but that the whole concept of the professor is part of the problem. And so in my chapter, I’m thinking about how can we think about not just having different kinds of bodies and people occupying this social role, but what ways do we need to actually change the social role in the first place? And so I think that the kind of epistemic authority that a professor has is often this individualistic, like knowledge is a kind of property that is sort of won through this genius and hard work or whatever, and not thinking about the ways in which knowledge is constructed and maintained in communities. In my chapter, I’m thinking about how do we teach in a way that presents academic or professorial or epistemic expertise in a way that acknowledges the ways in which knowledge comes about in and through communities. And so the vulnerability and metacognition are sort of like two strategies. So for me, like kind of leaning into the vulnerability and modeling epistemic humility when you don’t know things and being engaged in the process of coming to know rather than seeing knowledge is something that you arrive at, like “Now I’m a professor, so I know all the thing and you are the students and you don’t know the things and let me figure out a way for you to have the knowledge that I have,” …because even in, I think, constructivist or collaborative models of education, I think there still tends to be this like, individualized aspect. And there isn’t always like a true sense in which the space of the classroom is co-created, co-constructed by the community that we are all a part of… the professor and the students. That’s the sort of overall approach that I took to the challenge of the question.

John: How do you sell this to students? Because that’s an approach that they may not be familiar with based on their past educational experience.

Rebecca S.: Yeah, this is always, I think, a really big challenge, especially when you’re trying to do something that’s sort of radically different, and you want to do it in 16 weeks, and they’ve had years and years of education going against it. I also do things with games and play and game-based learning. And I actually have found that cultivating an attitude of playfulness can go a really long way towards breaking down some of the ways of being that students have been sort of trained to be in in the classroom. And so I think that there’s a way in which, if I can open students up to laughing and having fun, and just getting in a different kind of physical and mental space, then can sort of start to chip away at some of that. And I don’t think it’s possible to do in one semester. This is another thing about the problems with stereotypes. There’s not some magic pedagogy that’s going to eliminate racism, you have to accept the limits of what we can do in one class as one instructor for one semester. [LAUGHTER]

Rebecca: I really like some of the work that you’ve been doing related to games in the classroom. And we’re hoping that you might expand upon that a bit. And one of the things that struck me about… I think it was a blog post that you wrote about Dungeons and Dragons in the classroom… that you talked about how the people participating in the game, create the world and create the experience together, and that the knowledge of the space is created together. And it seems really tied to the work that you have posed in Picture a Professor. Can you tell us a little bit about how you’re using Dungeons and Dragons to teach ethics?

Rebecca S.: For me, when I first played Dungeons and Dragons, I was in my 30s already. I’m not one of these like lifers forever with DnD, but I realized how many parallels there were with the players and the Dungeon Master and the teachers and the student, because the Dungeon Master has a particular role to play, or the Game Master in a role-playing game. And it’s an importantly different role than the role that the players have. And I think the same is true, it’s not that the teacher and the student are the same in the classroom. But what’s really exciting about role-playing games, or at least tabletop role-playing games like Dungeons and Dragons, is that what the game is about, like what the values are… Is this going to be a game where we mostly focus on strategy and try to get as much money as possible to buy all the cool weapons or whatever? …like, that’s one way the game could go? Or is this like going to be a funny game where we’re just like joking around and getting into high jinks and whatever, are we going to explore some really serious things with identity and characters, like all of those things can be a Dungeons and Dragons game or a role-playing game. And it’s not something that can be dictated from the Game Master. It’s something that comes together through the creative, collaborative storytelling of the game itself. And so I think that there’s just so many parallels with teaching, where the teacher does know more things, and does have a responsibility to make sure that everyone’s involved, everyone’s included, everyone’s enjoying, everyone’s achieving the goals. There’s a certain responsibility of the teacher or the Game Master. And there’s a certain kind of knowledge and a certain kind of expertise. But what the game is, is fundamentally co-created,

John: Do you use that directly or indirectly to help share your teaching philosophy with students?

Rebecca S.: I do always try to share with students why we’re doing what we’re doing. And I’m trying to be intentional about making transparent, especially when we’re doing something weird.you knowe… I’m like, okay, alright, come on, like, you guys, humor me, we’re gonna try this thing. It may totally fail, but we’re gonna try it. So I’ll often talk to students about my teaching philosophy. I’m not sure that I’ve ever actually made that particular idea explicit, and maybe I should. So I think it’s more implicit. But now you’ve made me think that I probably ought to make it explicit.

John: Or maybe have them think about it and make the connections themselves.

Rebecca: Can you talk about how Dungeons and Dragons has unfolded in your ethics class, and what that assignment or activity actually look like?

Rebecca S.: So the way that I’ve constructed it now is that it’s the last four weeks or so of the semester. So it’s after we’ve covered a bunch of material. My dream is to figure out a way to make it the whole semester, but I haven’t quite gotten there yet. So we have several units and we study different philosophers, different ethical theories. So we do Aristotle and we do Mozi, an ancient Chinese philosopher, we do have Simone de Beauvoir and existentialists, they see a bunch of different ethical theories. And then for the last four weeks of the class, in groups, they create a character based on one of the philosophers that we’ve studied or one of the theoretical frameworks that we’ve studied. And so they have to come up with a backstory, like what kinds of experiences would someone have that would lead them to have an existentialist ethics or like, what kind of a person would be a Kantian and they have to pick a class. So they’re like, would Beauvoir be a wizard or a Paladin or whatever. And so they have to pick a class. And then they create their character. And in Dungeons and Dragons, there’s an alignment system. I have my own kind of alignment system, so they have to say whether they’re more focused on contextual factors or universal principles, that they have to pick their alignment for their character. And then we play the game. So I describe a scenario and then they have to say what choices their characters would make in a given situation. So I’m like, okay, you’ve all been called here by the Queen whose son has been kidnapped, and she sends you off to go rescue her son. So they’re all like, “Yes.” But then as they’re leaving the towns, and people say, “Actually, we don’t like the queen, we don’t want a monarchy.” And then they have to decide like, “Okay, do we save the queen’s son, which we made a promise to do, but that’s going to perpetuate the monarchy or do we help these rebels who want to bring in democracy.” And so there’s different decision points along the way, and they have to decide what their characters would do. And then they write reflections at the end of each day of the game, where they say, what decisions they thought fit well with the ethical theories and which ones they think could have been better and why. So they do some post-game reflection.

Rebecca: How have students responded to this kind of an experience?

Rebecca S.: So I’ve done some surveys and things just to ask students, and I don’t think I’ve really gotten anything really negative, everyone seems to think it’s fun. They say that it’s really helpful as a review and as application of the ideas. Some of the students get super into it. So that’s always fun. And I think that’s true with any activity you do. Like, you’re always going to have those students that are just like, “This is my thing” and run with it. So that’s really fun. And those students are not always the students that are necessarily the most engaged in the other parts of the class. And I think that’s really a benefit too of doing these kinds of different sorts of activities, because you have a student that maybe hasn’t had positive experiences with academics so far, but they hear Dungeons and Dragons and they’re just ready to go. And then even the students that are not as familiar with it, they have fun with it, too. And some of them kind of get into it unexpectedly. I also don’t force anyone to play. So one representative from each group plays each day, but I always make sure everybody has the chance to play, but they have the option if they just want to observe and do the reflections, that’s fine. So I think that helps with some of the potential discomfort that some students might have. But what I find overall is that the humor of it is really interesting, and really solidifying of both knowledge and community. So for example… I think I talked about this in the blog post, because this is just my favorite example… but I had a student who was playing a character inspired by Kant, and one of Kant’s principles is you’re never allowed to lie. And so they were sneaking into this goblin cave and the Kant characters, like, “I can’t sneak, it’s a lie, it’s deceptive.” So he goes in and announces that they have arrived at the goblin cave, and everybody laughs because everybody’s in on the joke, because everyone knows that Kant says, “You’re not allowed to lie.” And so there’s this kind of inside humor that is possible. That really is like, “Oh, we learned something this semester, we all now get this Kant joke that, at the beginning of the semester, no one would have understood.” It creates this sort of in-group thing, but not in a negative way. But like a positive way, like we’ve all learned this together. And we have this shared humor now. That I think is really fun.

John: In one of your tweets, you mentioned that you were planning a course that would involve some world building, could you tell us a little bit about that?

Rebecca S.: So I agreed to do some pedagogy through world building for this book that’s going to be coming out. It’s for a case study for the book. And it may or may not actually even be included in the book, but a bunch of faculty are doing some world building and writing about it, and then we’ll see what happens with the book project. So I agreed to do that. And then I’m also doing this class for the Social Justice Studies program. So I was like, “How can I match these up?” And so I was thinking about how to teach with world building, and the first thing that came to my mind is Plato’s Republic. So Plato’s Republic is a dialogue by Plato where he is exploring the concept of justice and he imagines this ideal city that ends up not being ideal in the end, but he’s trying to envision what would justice look like in this city and so he creates this world with its own myths, and with its own laws and rules and education system and marital practices. So I was thinking that it would be fun if we started with Plato, and then had students create their own just world where we think about what is justice? And what would justice look like in all these ways. This is the plan so far… there’s a site called World Anvil that is a world building site that people use for making role playing games, but also novels and things like that. And it’s kind of like a wiki. So essentially, the students are going to be creating a wiki of their world. And so we’ll have to decide what we want to focus on, because obviously, we can’t do everything. So we’ll have to think like, do they want to talk about education systems, do they want to talk about criminal justice systems, do they want to talk about religion? So they could come up with their pantheon of deities, if they want. What would religion look like in a just society? Would everyone have the same religion? Would there be no religion? The topics will be student led, I’m going to have a list that they can pick from, then the readings will be determined by like, we decided to do religion, and I’ll give them some readings, and then the assignment will be to build that part of the wiki, the religion part. And recently, I just started thinking, we’re not going to get very far in one semester. So what if this was like a project that the next time I teach this class, they take up, and we keep building the world. So, it may end up being a long project where each class picks up where the last class left off, which I think would be really cool. This is still in the planning stage,… the next couple of months to really nail it down. But that’s where I’m at so far. [LAUGHTER]

Rebecca: I really love the idea because it involves students in a lot of decision making, and really contemplating the ideas about social justice. So they would need to have some background knowledge, and then have discussions and co-create and co-decide on things and so it seems like a much more active way of engaging with the material, then maybe a traditional paper or other kinds of activities, like quizzes and stuff might have.

Rebecca S.: Yeah, and I think having this project, it’s kind of problem-based learning, in a sense, I guess. It’s a little bit different. But having that sort of shared project, I think, also hopefully will create a different kind of engagement and motivation.

John: It sounds fascinating. Can we go back, though, to your chapter in Picture a Professor? You’ve talked a little bit about vulnerability and how you share that. But could you talk a little bit about how you build metacognition into your classes.

Rebecca S.: So philosophy classes in general focus a lot on class discussions. And I think sometimes students think, “Well, what do I really learn from a discussion?” So going back to this idea of the professor is the one that has the knowledge, like, “Well, I’m here to learn from the professor and not from my classmates…” or they think that a discussion is just saying things, just whatever comes up, and I promise this is going to get to metacognition. And so what I focused on a lot is thinking about how to improve class discussions and how to help students think about what they’re learning from class discussions, by thinking about conversations in terms of academic moves. I didn’t invent this idea, but I’ve really tried to take it and develop it. So this idea that when you contribute to a discussion, you’re not just saying something, you’re asking a clarification question, or you’re posing a hypothetical, or you’re disagreeing with someone, or you’re agreeing with someone and saying why, or you’re connecting to your own experience, right? So there’s these specific things that you’re doing. You’re not just saying things, you’re doing things in a conversation. And from a social justice perspective, you can also do things like welcome someone into the conversation with a question or exclude someone or silence someone. So the idea that saying things does things is, I think, one of the most important ideas I want students to come away with. So I do a lot of work with having students identify what moves they’re making in a conversation and thinking about and reflecting on what kinds of moves are most productive, or are there any moves that we don’t want to make, like fallacy or ad hominem, like illegal move. But having students reflect specifically on what sort of contributions they’re making or how they’re moving a conversation forward. And this is not just for conversations, but also for writing, or you can also identify moves that people are making when you’re reading a text. And this is also a way of recognizing the communal nature of knowledge because one of the move can be to thank someone for helping you see something in a new way. Like that’s a move too, like showing gratitude to someone else, or acknowledging someone’s contribution or summarizing what someone else said or asking them to clarify. So there’s a lot of different activities that might be involved, but they’re all about sort of metacognitively reflecting on academic discourse, whether that’s written or spoken, and specifically identifying the ways in which we see through that, that this is us together creating a community rather than each individual person gaining knowledge on their own.

John: So how do you implement that specifically in terms of students reflecting on that? Do you have them engage in a conversation and then reflect back on their participation? Or are there other techniques that you use?

Rebecca S.: Yeah, I’ve tried a bunch of different things. So I’ve tried a game version, where it have the different moves on cards. And it’s an idea I got from Ann Cahill at Elon University. So she actually had a deck of these cards. And then I’ve tried that where I deal out the cards, and then they have to look for an opportunity to play their specific move. And that works kind of okay, but I haven’t had complete success with that, because students often find it difficult to find the moment for their card, or they want to say something and they don’t have the card for it or whatever. So I think that’s fun as a way of introducing it and practicing it, but it can interrupt the flow of the conversation some. What I did recently in an online class this past semester, is I actually divided the moves into different levels. So things like “connect your own experience,” most students can do that relatively right away. Whereas “identifying unstated assumption” like that’s a really hard move that takes a lot of work and practice. So for their first paper, I use Perusall, and so they comment on the text and respond to each other. But in their responses to one another, they have to identify the move that they’re making. So they’re responding to a classmate and they’re saying, “Okay, I’m going to agree and give a new reason. I’m going to disagree and explain why or I’m going to propose a hypothetical or whatever.” So I have them actually, in their discussion posts, identify the move that they’re making before they make the move. So that’s one way I’ve done it. And then I’ve also just done some things where I have them just write reflections on the discussion and identify moves that have been made or moves that haven’t been made, and why haven’t those moves been made? So things like that, also.

John: Do you have them use tags in Perusall for the types of moves that they make?

Rebecca S.: That’s a good idea, I have not, I only a very basic Perusall user. I was using Discord for awhile. And this is really fun, I actually had to make custom emojis for the different moves. So that was a fun activity at the beginning where they had to come up with an image to associate with the moves, it didn’t really play out fully in terms of the way that I envisioned it. But I think there’s still promise with that approach as well, I just need to pursue it more.

Rebecca: I like the idea of actively having to be conscious of what kind of move you’re making while you’re making it. That does seem like it may work a little bit better in an asynchronous environment where people have time to think about what move [LAUGHTER] they’re making, rather than in a synchronous context. When you were talking about conversations, it was reminding me of a really interesting conversation that we had, on our campus, with our workgroup on accessibility practices, with some students with disabilities, who identified that conversation, like classes that focused on discussion, felt really inaccessible to them or were hard to follow because they were having a hard time pulling out what to take away from the conversation. So you led this little segment about that. So it made me start thinking about how could we slow things down a little bit to be a little more cognizant of what we’re doing and maybe give time to digest what’s happening rather than the rapid fire and not being able to keep up. Although it maybe isn’t a natural flow of conversation, it does make you think more about what it is that you’re doing before you’re acting.

Rebecca S.: Yeah, that’s a really great point. Now that you mention it, I do think that it is often more successful in asynchronous classes. And in the synchronous classes, it’s often more of a after-the-fact reflection, but I think that it could be useful to have students plan out their moves in advance of the class. It’s like in preparation for the synchronous discussion and then, see then maybe not just like, here’s your card, make your move. And everyone’s like, “I don’t know, you have to pose a hypothetical right now?” But if everyone knew the moves that they were supposed to make in the class that day in advance, I think that would actually work really well. So I’m going to steal that and do that. [LAUGHTER]

Rebecca: Yeah, that might be interesting. I think one really good accessibility practice for any kind of presentation or activity is for people to know what’s going to be expected of them in advance so that they do have time to think so I’d be interested to hear, whether or not if you do that, how that plays out. So I know inclusivity is important to you. Can you talk a little bit about how some of the practices that we’ve been discussing today promote inclusivity? Or the ways that you think about setting up activities to make sure that people feel included?

Rebecca S.: Yeah, so I’ve been thinking a lot about inclusivity and creativity and vulnerability and playfulness. And I think it’s always tricky, because I think that sometimes we have this idea that there’s some sort of ideally accessible and inclusive class, that if we just keep opening and opening and opening and opening that somehow eventually it will include everyone equally. And I don’t think that that’s possible, because I think that when you really get into the concrete details of things, something that works really well for one student might be more difficult for another student, and how do you balance and weigh these kinds of complicated decisions? Nothing is ever just straightforwardly more accessible or more inclusive, I don’t think. So, I think it’s much more complicated than that. So the way I’ve been thinking about it is actually through a philosopher that I mentioned in a Chapter, Jose Medina, his book is called Epistemology of Resistance. And he talks about the need for epistemic friction. He talks a lot about the ways in which people who have a lot of privilege don’t encounter enough epistemic friction. So there’s this way in which things are too smooth and too easy, and you’re not challenged enough. There’s not enough resistance that you face if you have a lot of privilege. And he talks about the kinds of… I don’t want to say benefits, because that’s not quite right… but, the ways in which having a marginalized or oppressed identity can create the opportunity for developing certain virtues, certain epistemic virtues, not to say that it’s a good thing, like, obviously, it’s a bad thing. But what he does is he kind of flips things on its head where he says, we often think about privilege as benefiting people who have privilege. But there’s also ways in which privilege isolate you and prevent you from being able to know things or learn things or develop certain skills. So to get back to inclusivity, I’ve been thinking about being a teacher is about managing epistemic friction. So the idea that at certain times, certain students in certain contexts actually need a little more friction, and certain students need a little less friction. And so rather than thinking about creating some ideally open space, it’s about managing the kinds of friction that students run into. So some kinds of friction are unjust and should not exist. They’re not even friction, they’re just obstacles. But some kinds of things that are difficult or uncomfortable or challenging can be really good and beneficial for learning. But it’s really tricky to know which students need what and you always have to be very careful about presuming that you know what students need, of course, but I’ve been thinking about designing classes in terms of eliminating unjust epistemic friction, but creating opportunities for certain other kinds of friction, and thinking about accessibility in terms of that, like, is this a productive kind of challenge? Or is this an unfair or unjust kind of challenge? Because it’s not about whether it’s difficult or not. Learning is difficult. It’s like, is it the right kind of difficulty? Is it the right kind of challenge? Is it fair and just and promoting of the learning rather than the opposite of that, if that makes sense.

Rebecca: I like the word choice that used of obstacle versus a challenge. An obstacle is something that it shouldn’t be there… [LAUGHTER] versus a challenge is something that we would hope students actually do experience as students, because challenges can help us learn.

John: How do you set students up for this, because some of those discussions could be somewhat challenging for people who don’t want to have their beliefs challenged, who have really deep beliefs, and are resistant to learning new things, [LAUGHTER] or new experiences.

Rebecca S.: This is where I think that modeling some epistemic vulnerability is really important and humility. And so, for instance, I teach critical thinking, and I had this activity all designed, and I was excited about it and I thought it was gonna be great. And it was terrible. And I said to the students, at the end, I was like, “Well, that didn’t work at all. Like that was a disaster.” And then I’m like, “okay,” then it was like, “You guys, I have another class in 15 minutes. I have another section.” I’m like, “Quick, what do I do? How do I make this better? How do I save the next class from this terrible disaster?” It wasn’t that terrible, but it was just awkward and didn’t really work. And then they were like, “Oh, well, the problem is that you thought that we wanted to talk about these things. But we actually don’t care about these issues that you think we care about. We want to talk about these other things like ‘Do aliens exist?’ or whatever.” And I was like, “Oh, okay.” So in the next class, they ended up debating whether a hot dog was a sandwich. And I thought, this is so silly. Why are we talking about this? But they got really into it. And I told the next class, by the way, the last class sucked, and I changed everything. So here we go on the fly. And so I let them know that I made a mistake or I was wrong. I miscalculated how things would go and I made a change. And now we’re going to try it. And at the end of that class, there was this student and he had changed his mind about whether a hot dog was a sandwich. And he was really resisting admitting that he changed his mind. And so it was just like, I’ll call him John. John. His name’s not John. So I’m like, “John, you were wrong about whether a hot dog was a sandwich. He was like, “No.” John, you were wrong, right? And then everyone’s like, he wouldn’t say it, and I’m like, “Can you say ‘I was wrong?’” …but it’s really funny at this point, everybody’s laughing. And then finally he stands up. And he says, “I was wrong about whether a hot dog was a sandwich.” And everybody laughed and laughed. And I feel like it’s a really tiny way to admit that you’re wrong. But for me to admit times when I’m wrong, and then to celebrate students when they are willing to change their mind, even about something silly, it’s not going to get people to be totally open to changing their mind about controversial topics by the end of 16 weeks, or whatever. But it’s like, a little opening into practicing that habit of being willing to say, “Yeah, I was wrong about that.”

John: That reminds me of a podcast I listened to recently, I think it was Planet Money, or Planet Money Indicator, where one of the issues they were discussing is whether a burrito is a sandwich or not, because in New York State that affects its taxable status. So, it was actually a major political issue.

Rebecca S.: That’s so funny. I’m actually having my critical thinking students debate whether hot dogs are sandwiches right now in my asynchronous class, and actually, there’s not enough friction at the moment. They’re all like, “Yes, the hot dog is a sandwich, because it’s meat between two pieces of bread.” And I’m like, “No, the whole point is for us to practice disagreeing with each other.” So maybe I can throw in the podcast, or I can find some article.

John: I’ll include a link to that in the show notes. You discuss vulnerability a few times, might that be a little bit risky for a younger female professor in terms of the known biases that exists in terms of student course evaluations?

Rebecca S.: I think it’s really important for me to say that, yeah, I’m marginalized in the sense that I’m young-ish, female, professor in philosophy in particular, where there’s not a lot of women. There’s also things that I can do that other faculty can’t do. I have a lot of privileges as well. And I’m white and certain personality things, I think I can, like, get away with things. Everybody’s different. I don’t think there’s any wrong way necessarily, I’m not saying everybody should do what I do at all. I don’t think that that’s true. But at the beginning I was talking about what felt authentic to me. I think I needed to find a way to be able to be myself in the classroom. And so I don’t think that I can give some sort of universal prescription that will work for everyone. And I do think it’s risky. But I also think that the risks are unavoidable. I was saying before, that we’re not going to eliminate sexism or racism, oppression in the classroom, because our classes are part of the world. And those things exist in the world. So while I think it’s risky, I also think that it’s just risky to exist, [LAUGHTER] and that we are vulnerable, whether or not we want to admit it. This actually comes from Judith Butler, but there’s not really like whether we are or aren’t vulnerable. It’s just how do we manage our vulnerability. And so I think it’s not even necessarily about being more vulnerable. It’s just a different way of managing the inevitable vulnerability of being human. And being a teacher is super vulnerable. Being students also, humans are vulnerable, and that’s beautiful and scary. And so I probably framed it in terms of more or less, but when I think about it, I don’t actually think it’s about more or less vulnerability. It’s like, how do we manage it? Do we acknowledge it? Do we not acknowledge it? And I don’t think necessarily that we need to acknowledge it all the time. And different people are going to have different ways of thinking about it. But I do think it’s important to acknowledge epistemic vulnerability to a certain extent, because I think that it’s true that no one knows everything. And I think it’s harmful to perpetuate an idea that there is some sort of place you’re going to get to where you don’t need to continue learning. So I do think that epistemic vulnerability is important to acknowledge; other kinds of vulnerability, I think, may or may not be, it depends on the person.

John: And I would think it would also help to nurture a growth mindset in students when you acknowledge this epistemic vulnerability, letting them know that that’s just a normal part of learning, that there are many things they don’t know, but they can get there.

Rebecca S.: I share with them sometimes my own experiences of writing a dissertation and how difficult it is to get critical feedback. I don’t think necessarily everyone needs to do that. But I think that, for me, opening up a discussion where if I’m about to give them feedback on their writing, talking about what it’s like for me to get feedback on my writing and how that can be hard. And here’s sort of what I do when I’m about to read comments on something. Sometimes I need to take a minute. So, there’s ways of sharing that depending on your comfort level, but I do really think that acknowledging our humanity can be a really good and powerful thing.

John: We know you teach in a community college with a very high teaching load. And you mentioned you have a baby in the background there. But you also are playing with a band and you’ve released your fourth album, how do you record with a band and create music while also being a full-time faculty member with a heavy teaching load?

Rebecca S.: I’ve no idea. [LAUGHTER] No, I, you know, don’t sleep… No. Well, to be honest, and talking about, like acknowledging humanity and vulnerability, the last year has been incredibly difficult with a new baby. And it has been really, really, really hard. I have done it somehow. But I don’t actually think that the last year has been my best year of teaching, having a new baby. And I think that acknowledging that that’s okay, I’m still doing the best that I can, and things are going to get better now, both like daycare, and like, whatever, having a lot of support. I mean, you don’t do these things alone, also, I think. And so, I mean, I’m really lucky to have my husband who largely works from home. And he’s takes care of the baby a lot of the time, and he’s also in the band, and then having the ability to get a babysitter sometimes. So there’s a lot of ways in which the way that it works is the result of luck, and privilege and support. But then also, I think that if you’re doing things that you really care about, and that you love, you try to find ways to make it work and you just find ways to be more efficient. And like with teaching, one of the things I’ve realized is my impulse to want to reinvent everything all the time is not always what serves students the best. And so thinking about what I want to do, because I’m really excited about some new idea, sometimes doing the thing that I’ve done a million times that I know works is actually better for students and for me. And so thinking about that, too, like, my approach to teaching was always like, “Oh my God, every semester, I’m gonna do something wild and crazy and completely different, every single time.” And it’s unsustainable for me, but it doesn’t actually serve students. So I still do my experiments, and I think I always will, but I think being more deliberate. Okay, I’m going to take on this one project, I’m going to redesign this one class…

Rebecca: …and not like ten?

Rebecca S.: and not all of them.

John: And not redesign everything in the class, which I think Rebecca and I also have a tendency to try to do.

Rebecca: Yes, I’m trying to be more sustainable, my new approach.

Rebecca S.: And I think that oftentimes, it is better for students, I think that sometimes it might not be exciting for me, but it’s their first time experiencing it. And so I think that’s important to keep in mind.

Rebecca: Or you rotate between the things that you’ve invented, so that you stay interested, they’re still well established.

John: And the second time you do things you often have learned from past attempts at doing them, and they often result in better learning outcomes for students.

Rebecca: So we always wrap up by asking what’s next?

Rebecca S.: Yeah, so I guess I’ve already talked a little bit about what’s next in terms of like this world building class I’m working on. I’m also writing a bit about my role-playing game for ethics. I’m also working on something more about cultivating playfulness. So I’m interested in thinking more about exactly what it is about playfulness that I think is so meaningful and important and how that can be serious play. So I’m really interested in thinking more about these connections between playfulness, creativity, and inclusivity.

Rebecca: That sounds really exciting.

John: It does, and are you working on your next album?

Rebecca S.: Not yet. It just came out June 10. So we take a long time to write songs. So expect another one in like four or five years. [LAUGHTER]

John: We were doing a little bit of research on your work, and I ended up spending a lot of that time listening to music while Rebecca was actually reading your blog posts. [LAUGHTER]

Rebecca S.: Nice. Awesome.

John: I enjoyed it, it was really nice. Thank you.

Rebecca S.: Thank you.

John: Well, thank you. It was really great talking to you.

Rebecca S.: Yeah, thank you so much for having me. This was awesome.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]