275. Improving Learning and Mental Health

 Student reports of mental health challenges have been rising rapidly for several years. In this episode, Robert Eaton and Bonnie Moon join us to discuss what faculty can do to better support students facing these challenges. Robert and Bonnie aretwo of the authors of Improving Learning and Mental Health in the College Classroom, which will be released later this spring by West Virginia University Press.

After completing a law degree at Stanford and working for several years as a litigator and general counsel, Robert returned to academia in 2004 as a member of the Religious Education faculty at BYU-Idaho. He is currently a professor of religious education and a learning and teaching fellow, and has previously served as the Associate Academic Vice President for Academic Development at BYU-Idaho. Bonnie is a member of the math department at BYU-Idaho, where she also serves as STEM Outreach Coordinator.

Shownotes

Transcript

John: Student reports of mental health challenges have been rising rapidly for several years. In this episode, we discuss what faculty can do to better support students facing these challenges.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Robert Eaton and Bonnie Moon. They are two of the authors of Improving Learning and Mental Health in the College Classroom, which will be released later this spring by West Virginia University Press. After completing a law degree at Stanford and working for several years as a litigator and general counsel, Robert returned to academia in 2004 as a member of the Religious Education faculty at BYU-Idaho. He is currently a professor of religious education and a learning and teaching fellow, and has previously served as the Associate Academic Vice President for Academic Development at BYU-Idaho. Bonnie is a member of the math department at BYU-Idaho, where she also serves as STEM Outreach Coordinator. Welcome Robert and Bonnie.

Robert: Good to be here.

Bonnie: Thank you.

John: Our teas today are: …are either of you drinking tea?

Bonnie: [LAUGHTER] I brought my lemon water. Can I still be on your show? [LAUGHTER]

Rebecca: Well, they are two key ingredients of tea.

Bonnie: Yes, right. That’s what I thought.

Robert: And I brought my favorite flavorful herbal tea. Sweet and Spicy Original from Good Earth.

Rebecca: That sounds good. I’m sporting the Hunan jig again, John. That’s all I got. [LAUGHTER]

John: And I have a ginger peach green tea today.

Bonnie: Oh, that sounds delicious, too.

John: We’ve invited you here to discuss Improving Learning and Mental Health in the College Classroom. Could you tell us a little bit about how this project came about?

Robert: It actually started with me. And it’s hard for me to trace exactly when it started. But I’ve been out of the classroom on a full-time basis for a while. And when I got back in, I was amazed to see just how many students were flaming out and fizzling out by the end of the semester because of mental health challenges. And I’d sensed that this was an issue of increasing severity, but still seeing it firsthand, especially after a few years away, was really breathtaking, and got me thinking about the way that we teach and our course design decisions and what effects that might have on students and whether there were things that we as professors could do. So I ended up kicking off a semester-long faculty learning community exercise, we call them a “Think Shop” here and Bonnie was one of the members of that group. And I thought I wanted to tackle a book, and eventually as we got into it, I invited Steve Hunsaker, who’s not with us today and Bonnie to join us. And it’s been a marvelous collaborative effort.

Rebecca: Can you talk a little bit about who your target audience for the book is?

Bonnie: College professors, somebody like me, who didn’t come from a background heavy in psychology or understanding the psyche and to someone that loves mathematics… someone like me, who doesn’t really understand all of the details, but wants her students to feel safe in her classroom and have a safe place to study and to thrive and to be passionate about something. And so, yeah, so college professors, research or at our Institute, we focus on teaching but either kind of Institute.

Robert: So that said, our research assistant has now graduated and is teaching elementary school, and said she uses many of the ideas from the book. And they’ve been relevant even in a K-12 setting. I was thinking through how we probably, from a marketing standpoint, didn’t choose wisely enough… that really many of the ideas in the book would be beneficial to a student, even if their teachers choose not to do any of these things, they could still realize, “Wow, community matters, maybe I should try to connect with some people and create a study group, even if the teacher doesn’t facilitate that.” So I would say college students with mental health challenges and their parents might benefit from it as well,

Rebecca: When we’re running professional development, which is just another setting of teaching, those students might also fit some of these descriptions, too. [LAUGHTER]

John: Could you tell us a little bit more about that faculty learning community that this work began to grow out of?

Robert: A few years ago, we’d started a formalized faculty learning community exercise, we branded ours a “Think Shop,” but basically, it’s a semester-long effort where faculty from different disciplines come together and take a deep dive into the scholarship of learning and teaching or something that affects teaching and we exchange ideas. So we met every week, we each read different things, and I kind of facilitated that and led a discussion. So it was delightful to get to know colleagues better and to brainstorm and benefit from the different disciplinary perspectives. Bonnie, would you add anything to that?

Bonnie:Yeah, I am so grateful that I found that community. Not only are we hopeful that we’ll create a place where students can thrive, but I think it’s important for the professors that run these classes to also model this and to participate in these things. So having that community with other colleagues, especially right at the start at COVID, [LAUGHTER] was very helpful to me and my mental health. And it was just invigorating to learn new things and understand a whole new discipline that I had never studied before and to try and understand what’s happening. So I think it was an opportunity for me just to even model when I’m hoping that my students will do as well.

Robert: It turns out community is not just good for students from a mental health standpoint, but for faculty. For us, the timing was fortuitous, it came right on the front end of the pandemic. So at the time when people were kind of having to withdraw socially we continued it virtually, and it gave us some great community support.

Rebecca: It’s probably worth noting that you just mentioned that this conversation started prior to the pandemic. Certainly, our awareness around mental health issues have been raised and related to the pandemic, but these were growing issues among college students prior to the pandemic, for sure. Can you talk a little bit about why we’re seeing these mental health challenges so prevalent among college students?

Robert: It may be a cop out that we took in the book. But we basically said, what are the root causes of this? We’re not really sure, but we know they’re arriving, and that the problems are real. In fact, one of my fears, sometimes, with the conversations among some of our more hard nosed colleagues, when we talk about roots is that I feel they’re a bit dismissive of the symptoms, and think, well, people just sort of buck up, and if they’d put away their cell phones and move some pipe and work like we used to everything would be fine. I’m fascinated by the debates. And in fact, there was one item, we were going to include in Bonnie’’s chapter, and then a meta analysis came out that was contrary to the other studies that I’ve been seeing. And so we left it out. So I’m puzzled, intrigued, and have my own guesses. It’s hard to ignore technology and the way it’s changed society. So that’s certainly a controversial but leading candidate, but something has changed. There’s a little bit of people being more willing to go and get diagnosed, and maybe a little bit of change of measurement. But there’s some pretty good solid measurements, like attempted suicides or self harm, where it’s not just categorization, where we can see this is really snowballing over time. So our short answer is, we’re not really sure. But we know they’re coming to us with these problems, we are rooting for those people researching the root causes. And we’re kind of leaving that to others as we deal with the symptoms that we see in our classrooms.

John: We’ve always lost a lot of students along the way. And some of that seems to be related to the stress and anxiety that students experience. You describe in your book, the high-risk, high-stakes, environment of college. Why would that tend to increase the prevalence of mental health challenges for students?

Bonnie: So first of all, they’re just coming from maybe a secure place, high school, a place where their teachers were there, they knew them every day, they were very structured, and they knew their schedules, and then they’re coming to this new place to navigate. And it’s a whole new world. And so they’re on their own, their support system is not necessarily in place as they come into this new world. And it’s just like we said in the book about this petri dish just ready [LAUGHTER] for something to happen that does not seem normal or good. And so I think that the uncertain times of it, the deadlines that we put on them, maybe sometimes just not even having a friend. Some of these kids get isolated in their rooms, and they don’t see people for days, and then the teachers start to hopefully miss them. And I think that is part of where we can make a difference. And we were probably going to talk about this later, but when you think about the college experience, the teacher, the professor, is the one that has the most likelihood of seeing these students most often and most regularly, and when we don’t see them. I’m hoping that we’re alarmed. And like, “Where have they been for a week or two weeks or three weeks?” One day, I was in class, and I noticed one of my groups was conversing and that one of the students was really struggling to socially interact with these other kids. And he was just upset and mad. And the other three students were very uncomfortable working with him. And so I started to wonder what was going on. And I made a few phone calls, and one of the students asked me to please call the services on campus… I’m not sure what they were called back then, we’ve been working on this at our campus to get these services more upfront… but I think I ended up calling security because the kids were so worried about this kid, that he was going to be violent, that I ended up calling security. And so security actually had a program where they reached out to him, they went to look for him at his home. And the sad thing is that this kid never showed up to my class again. And so after that experience, I was like “What happened?” like do inform us of things going on. And I found out later on that he had passed away, and there was no details, there was nothing. And as a professor, I was like, “What could I have done?” I didn’t understand, I knew something was kind of strange, but I just didn’t understand, I didn’t have the tools, and it’s not our job to fix these things. But to be able to recognize something in our classes and get these students to places where they can get help. I think that’s something we can do, and it doesn’t go too far outside of the reach of our classroom. We’re trying to build curriculum, we’re trying to build awesome experiences that motivate students. A lot of us might say, we don’t have time for this when I have time to worry about the students outside of our classes. But it would have taken just a couple phone calls, and I know they did reach out. But I don’t know, I don’t know what happened to that student. But still, I feel like I can do something. Even if it’s small I can do something for one person. It matters.

Rebecca: I think the reality is that we often say we don’t have time yet we expend a lot of energy actually worrying about our students, or at least a good portion of faculty do, because they are missing or something seems not quite right and we don’t know what to do. So that energy is being expended whether or not we’re actually acting on it.

Robert: Yeah, in fact, I think there are a couple of false dichotomies to be aware of: one is I either do the stuff I’m supposed to do as a professor or I babysit kids with mental health challenges, or I either focus on being a high expectations professor who really helps students master the content or I just coddle them. And we find those both to be false narratives. For example, I now on the first day of class do things differently than I did the first 10 years. And I have students in teams, I make sure at least one person in the team has already read the syllabus, maybe taking the syllabus quiz, and then I have them show each other. First, I have them connect, get to know each other, share phone numbers, and then I have them show each other all the stuff. And once they’re all done, then I say, “Have you got any questions?” And there are usually relatively few. The dynamic on that first day is fundamentally different and better than it used to be and I get far fewer follow up emails and phone calls asking how to do something because they text each other, they know how to do it. So it’s a simple technique, that’s actually a great one, for helping our students who come to our classroom with some anxiety and wondering if they’re gonna have any help or be able to make any friends. And it actually takes me less time in the course of the next couple of weeks because of the fruitfulness of that investment.

Bonnie: Yeah, what if we could be better teachers, and help our students improve their mental wellness at the same time, and it didn’t take any extra time? Wouldn’t you want that recipe?

Robert: In fact, that’s why we went back and forth on the title, we really struggled. But we wanted to convey the notion that you don’t have to choose between improving learning and mental health, that really virtually every tactic we recommend in the book, we would recommend even to someone who somehow had no students with mental health challenges in their classroom. It just makes for better learning, they happen to also make life much better for students with mental health challenges.

John: One of the really nice things that I observed in reading through your book is that so many of the practices, as you said, are things that are recommended by people who study effective learning techniques. And one of the things you talk about is replacing high stakes exams with lower stakes activities. Could you talk a little bit about that?

Bonnie: Given a little bit away, I’ve been a student myself for the last two years. I took a sabbatical and went back to school. [LAUGHTER]

Robert: And what degree are you pursuing, Bonnie?

Bonnie: And so I love mathematics, but I wanted to see how math works inside of a nuclear reactor, so I went back to school in nuclear engineering. And so, [LAUGHTER] just coming fresh off of that, and having worked on this project with Robin, Steve, I’m like, ”oh, I have so many ideas for you guys about how you could help me not be so stressed out, and not so anxious today.” [LAUGHTER] They didn’t want to hear that. But I was like, “I have some ideas.” But one thing I think that I’m taking back to my classroom from that experience is choice. Like when my professors gave me choice, and they let me follow a path that I was passionate about, I was all in, I didn’t have to be pushed, I was pulled into that direction. And so when I think about assessment in my own classes now, like coming back, and I think about that final exam, and sometimes that is a high-stakes place, and very stressful, even all of our exams can be that way. So for my differential equations course this semester, and I have done this a few times before, I give a choice, of course, between a final exam or final project. And we start talking about this early on. And not all of them want the final project because it is a lot of work. But watching them light up, and to see that they could do something different than sit down and take a test for three hours.[LAUGHTER] It’s just so heartwarming to watch them, and it’s helpful to them. And I was telling Rob and Steve that I had one student that took the project and found something to do inside of his other classes he was taking, and he kind of connected them together. And to watch his passion and to see him come from the student that sat in the back row that seemed mad every day, [LAUGHTER] and when I mentioned this final project, he just lit up and I could see hope, and at the end of the semester, he couldn’t stop. Like I finally said, “You can get some sleep tonight, you don’t need to work on this every minute of the day.” It was just an amazing transformation to see purpose, help him come pull out of this. I don’t know if he was suffering from depression, but he was definitely down a lot in my class, and maybe that was the subject because mathematics tends to make people anxious sometimes anyway, [LAUGHTER] but just to see the turnaround. So I mean, I don’t know that we always have to put everything in one place, but we can give choice and let them kind of have some room to navigate their own way through our courses.

Robert: I should confess that I started from a pretty old school hard-nosed mentality. And that law school, it was a game I played well, so I thought it was a good game. And the in-class instruction, by the way, at my law school was fabulous. I still think that. But the course design, now I look back on that and I think a single assessment that’s three hours long at the end of the semester, and your feedback is one number. So I might have understood a few concepts really well and others really poorly. I actually have no idea. And for that matter, the professor has no idea how well the class is understanding things until she grades the final. She doesn’t have a chance to correct. If I were redesigning law schools, I’d say break it up into four tests. And then make it comprehensive. Give students the opportunity and incentive to fill in their knowledge gaps that they identify on an initial test or assignment. I’m just so embarrassed to admit that that, until I really studied for this and dug in and researched, really didn’t cross my mind. What happens when I give a test to students and many of them bomb it and we just move on. What am I hoping that they’ll learn? …to work harder for their next test? Well if those concepts were really as important and foundational as I claimed they were, I should be more concerned about finding a way to encourage students to go back, fill in those knowledge gaps. And so now I’ve softened up and I give them opportunity and credit, I still give them incentive to try to learn it right the first time, but I’d like to give them some incentive to go back and learn things they crashed and burned on the first time around so that they don’t get left behind.

Rebecca: You mentioned at the start of the conversation that there’s symptoms that we see, we don’t necessarily know the root causes, but we’re seeing the symptoms of various mental health challenges. And those symptoms impact student learning. So there’s a consequence to that. If we’re having anxiety, then we might be presenting that in a particular way. And then that’s probably impacting how learning is happening for us. Can you talk a little bit about what that looks like? And then we can follow up with “How do we support students who are facing those challenges?”

Bonnie: I think, especially in my math classrooms, I see a lot of stress, anxiety, and the way that I saw it this last week… I’ll just tell you that example. So we were talking about filling in the learning gaps. I do a little bit of just-in-time review so that my students get prepared for class because I understand that when there are those learning gaps, it can be very intimidating to come back to a class and try and start over again. But then as you start to see their stress… and today, the student just started to get angry at something that was going on in my class. And I was like, “What are you talking about?” He said, “Well, my last class, we got to use technology, and we got to do all these things.” And I’m like, “That’s okay.” [LAUGHTER] So I could see that he was stressing out. And he may even have… I don’t know… this was just the symptom, I think, of a deeper identity with him. Like he didn’t identify with math, he didn’t see this math, he saw this as something that he just needed to get through, and stressed people don’t do math very well. [LAUGHTER] And that might be true for other subjects as well. But some of the research I dived into showed that when you’re stressing out and the teacher is coming down on you that you can’t do math, so what are we doing in our classrooms? So I started to rethink things today and said, “What is the root cause of this? What is he really stressing out about?” And that was just a one time thing. But I think maybe you’re asking more about how we identify kids, it’s just not stress, but when they start getting depressed, and when they start having this anxiety. I mean, these are things I think sometimes that are harder to see, unless you’ve been there, or you had experience with that. So one thing that I will share with you is that when I had a situational depression, that was triggered really quickly, I was not prepared for it, I started to withdraw. And I started to come away from my life. And so I can recognize that more easily now, when students start to withdraw from their groups, communicating, or they start to miss class for two or three days in a row. Or they communicate in ways that you really aren’t socially kind, like sometimes they get mad. It’s not just madness, but sometimes they just become very withdrawn and apathetic. I mean, those things are normal for everyday. But if it continues for like two weeks or three weeks, then I start to say this student might be suffering from something that’s depressive or possibly anxiety. So I do look for things like that.

Robert: Let me just add to your question, Rebecca, anxiety, or stress can be a bit like the bull and pit. It can be advantageous sometimes. it can save our lives, when some adrenaline is needed, but in certain situations, and for certain people. In fact, it seems to affect different people differently. It’s very unhelpful in the learning process. So the consensus of the research that we saw suggested that maybe in short, occasional doses, it can be fruitful, but chronic and high doses, stress almost always interferes with the learning process. And then depression, it was just always bad, [LAUGHTER] we just couldn’t find anything saying this was ever helpful to the learning process. And there’s a bunch of physiological explanations for it. But, in fact, one thing to remember is, some of us do well, like I speak better, with some stress than without it. And so those of us who succeed in teaching as careers are probably people who dealt well with that anxiety and stress built into college. And it may make it difficult for us to understand and empathize with students who process that differently. And it may cause us to be just a little callous and say, “Yep, college is stressful, buck up.” But it’s helpful if I then think about how I sing a solo. I sing much worse than I sing in a duet or a quartet. And in practice, I sing better every time than I do in a public performance. [LAUGHTER] So in that area, I can see: “Wow, stress really undermines my performance.” So it’s just a matter of being mindful of how these mental health challenges, generally, when there’s too much of them and they last too long, almost always interfere with learning.

John: What can faculty do to try to create a more supportive but positive environment for the students, which will reduce extreme levels of stress and reduce anxiety which can interfere with learning?

Bonnie: So this is where I’ve been working,[LAUGHTER] coming back from my sabbatical and seeing and feeling the stress. I want to be more aware. And so some of the things that we researched and looked into, I’m starting to use more regularly in my classes. And so one thing for sure, and we hear this a lot, just as a regular teaching strategy is to learn their names as soon as possible when they know that somebody knows them, and somebody understands that they’re not there or they are there, it makes a difference. So this is our first week of school, and so I have put a lot of energy this week into taking those names home and learning their names. And I have an every day classes, this is day three in that class, and just being able to go up to them today and talk to them, ask them their name, their major, and even before class, I’m there trying to understand who they are, a little bit about their background. And so it’s important to me, it’s important to me when I was a student, that they knew my name, and that they knew I was Bonnie Moon, and that this was what my dream is and I’m hopeful that this will happen. And so I think that’s one specific thing that we can do. I realize that some teachers and professors have these huge classrooms [LAUGHTER] with 500 students. So, we’re very lucky here because we have classes between 30 and 50. And so we can learn their names in a couple of weeks if we work hard at it. So I’m very intentional at that at this point.

Robert: I’ll add another just in terms of messaging, we found when we did focus groups that students had had a variety of experiences with professors. So they were looking to read us, they’re looking to see what does this professor really think about my situation and do I dare go ask for a bit of flexibility if a mental health crisis arises. So I added two paragraphs to my syllabus, and I have yet to have anybody I felt like was trying to exploit it or take undue advantage of it. And I’ve had other students thank me who I had no idea had mental health challenges, and they didn’t have to use it. But they said, just knowing that I felt this way, put them at ease. I have this section on mental health challenges:

A growing number of students experience mental health challenges to varying degrees. Doing what you can to stay ahead and on top of depression or anxiety by wisely taking care of yourself will be a key to succeeding academically. But even then, sometimes these challenges can affect your ability to complete the required work. Or a particular assignment might trigger anxiety for you in ways that I’ve not anticipated. Or maybe you reach a point where you just can’t get yourself to class at all. In any of those cases, please come and talk with me, or at least send me an email. I’ll listen and do what I can to help. But the sooner you share your challenges with me, the more I can help. To learn the material and pass the course or earn an A you’ll still need to do every bit as much work as other students…

By the way, as an aside, all the students in our focus groups were not only fine with that, they wanted that. They didn’t want us just to write off assignments that they missed from two weeks that they were in bed with severe depression, continuing:

…but we may be able to find some creative ways to help you do that, especially if you approach me when your problems arise, instead of at the end of the semester.

I think I get a lot more students now willing to come in, my having made it safe through this provision, and let me know about their problems while they’re still in progress and we can still do something about it. Before I was getting a lot more coming in the last week, who were in a hole that was just so deep, I couldn’t, in good conscience, find a way for them to get a passing grade.

Rebecca: So the key is catching them before they disappear.

Robert: Yeah, I think doing some preventive things. So being proactive. We found some interesting studies about just mentors. And as students being less likely to commit suicide if there was an adult they felt like they could talk to about personal things if they needed to whether or not they had taken advantage of that. In fact, for almost everything, the leading intervention that we could find was improving that connection between professors and students. Whether you want to help more students graduate, more students thrive after they graduate, Frankly, even more students participate in class discussion. For all of those different outcomes, the single best intervention seems to be strengthening that connection. So we’ve shared some ideas in the book, and I might just share from our wonderful friend and colleague, Steve, who can’t be with us today, his thought. He says:

This takes me back to Uri Treisman and his amazing work. Treisman, who teaches at University of Texas-Austin tells his students they can succeed in calculus and that they belong, but he goes far beyond asserting that. He traces for students a mathematical genealogy in which they appear at the end of a long chain of ancestors that begins with Leibniz and Newton. He invites his students to meet with him on Saturdays for one-on-one conversations that may not be about calculus, but which are clearly about their success. The depth of Treisman’s heroic dedication to students astounds me, I may never get close to his level of commitment to his students, but I’ve taken a step in that direction by building one-on-one conversations with my students into the semester schedule. I believe that students understand that time is precious and that even 10 unhurried minutes of unscripted conversation about their plans, challenges, and dreams send a clear message about care and commitment.

So, that’s from Steve Hunsaker, our wonderful co-author.

John: So you mentioned both in your book and in the conversation so far that students do care about whether their instructors care about them. We’re not always very good at sharing that, though. I think most professors do care about their students, but that doesn’t always get conveyed. Certainly learning their names is one strategy. Meeting with them one on one is another strategy. And you mentioned, letting all the students know that they’re capable of being successful is one way of doing it. Are there other strategies that faculty could use to let students know that we do care about their success?

Robert: I’ll start with the baby one, if I might, and this one hurts, because even after presenting and teaching about it, I still catch myself doing this. I’m busy. So when I get an email that says, “I’m going to have to miss class on Monday for a funeral, is that absence excused? Or if not, is there anything I can do to make it up?” I tend to go right to “Oh, that is an absence that can be excused. You get three excused absences after that you can make it=…” And then once in a while, the thought will come to my mind, did you catch the word funeral in the email, they’re going to a funeral. They’re a person who’s going to a funeral. So I’ve tried to stop and say, “I’m so sorry to hear that someone you know and love has passed away? Do you mind if I ask who?” And they’ll email back, “It’s my grandma.” And I’ll say, “Tell me how has your grandma blessed your life? How are you like your grandma? What will you miss most about your grandma?” It takes me like 10 seconds [LAUGHTER] extra typing, but it converts what was a transactional email into a human email. So just to be human in our interactions with students, I think, goes much farther than we might imagine.

Bonnie: I’ll add to that. I think accessibility is something we can build into our lives as we look at our semesters, when we’re accessible to students, and we really do meet our office hours or we arrange to meet with them and that we make ourselves available. I think that sends a message that we care about their success and about them. But I think that’s something to do. I know that one semester, my stat students created a project, where they just said “Now are professors really in their classrooms, if they had a question? And so they went around campus during office hours and checked to see if professors were there. [LAUGHTER] And they had a great project. And they found out that their alternative hypothesis that professors are actually there less than they say turned out true. [LAUGHTER] So we had to do some work. But they cared, they care whether we care. So I think accessibility, and then the one thing I would add to that would be how we structure our courses, like we don’t have to go way out of our way to make this happen. We can restructure our courses so that we get the learning done. And actually, we can maybe even improve the learning as we restructure. So an example would be in one of my classes, I have a lab day built in. And it’s not a lab day outside of class time, it’s not asking them to go get in groups outside of class time, I actually create a lab day during the week that we come together and they get to ask questions, they can talk about the homework, they can work on their group projects, because I care that they have a life. I know my class is not their only class and meeting up with groups is difficult. And I care about that. And so I just build it into my curriculum. And so I do a few more videos, I do a little bit more writing so they have some things to prepare for class. If I don’t need to say it during class, I can put it outside of class. And then during class, we can use that time to collaborate and to foster relationships and to think about deep things and to get passionate about things. Because I’m there, the best time for them, I think, is with me. [LAUGHTER] I want them to be there and I want to be their tutor, I want to be the one that sees how they’re doing on their math problems. When they run into a hard math problem. I don’t want them going to the math lab, asking another student that’s at their same level the question. I want them to come ask me. [LAUGHTER] So I set up a day every week, and that’s what we’re going to do on Monday, it’s going to be lab day. And it’s kind of a nice breather after the first week, because I’ve kind of pushed them a little bit getting started, we get right into the mathematics. And then on Monday, we have a lab day, they can breathe and I can talk to them about how things are going, I can kind of assess how I’m doing with the teaching. And if I need to change things around for the next week, we can build it into our classrooms.

Robert: I’ve started using Calendly or then I moved to Bookings, but to make it easier for students to access me. And so it’s just anytime that’s available on my calendar, they can meet with me. And when they meet with me… I stumbled on this last semester… I’ve said have you got your phone with you? Of course they have their phone with them. Would you mind showing me a photo or two or a video that would help me better understand you. So this morning, a student shows me a fascinating photo of him and two friends and his snowmobile and a big hole that he’d gotten stuck in, and told me about his love for snowmobiling. I will remember him better and understand him better because of that. I’ve been amazed at the things that students have shared with me and how understanding their backstories changes my perception. I remember asking one student “Just tell me your backstory.” He said, “Well, I was abandoned by the side of the road, I guess because I had a cleft palate. And then I was in an orphanage until I was adopted.” He was in another country. Wow. This was a student who sometimes didn’t stop talking as soon as I would have liked him to stop talking after we’d done a small group discussion. I just saw him in a whole different light and was amazed by the things that he was accomplishing. So understanding students’ backstories, I think, helps strengthen that connection we have with them.

Rebecca: One of the things that I really appreciate about the examples that you’re sharing are that none of them are big time commitments. None of them are huge asks, but they’re cumulative when they add up, and they add up in a way that really demonstrates care, and then when I’m doing it, and John’s doing it, and Rob’s doing it, and Bonnie is doing it, then the student really feels supported.

Robert: That would be our dream, the more people who do these kinds of things, the greater the support network for our students. And you know, once in a while, we’ll do some things that we wouldn’t otherwise have done. On occasion, I will walk a student who is suicidal over directly to the mental health center, to the counseling center. And so it takes me five or 10 minutes, but at the end of the day, I feel like those were probably the most five or 10 important minutes of the day. Actually, that does remind me of one other thing. We’ve talked about connecting with the students, course design decisions, classroom tactical decisions, but we do play a role as gatekeepers. We’re in a position, not to provide mental health counseling, but to spot students in need. Students, when we do the kinds of things we talk about, tend to trust us. And then I’m surprised how many will describe what, to me as a layperson, sounds like depression, but they’ve never been treated for it, they’ve never seen a counselor. And so I just make the pitch. “Wow, I’m not a mental health professional, but I’ve talked to many students with similar symptoms who’ve gone to the counseling center, it’s free here, and wow, they’ve gotten some great help, and let me introduce you to some other resources.” So just being that wise friend who knows how to connect people with resources, we’re in a unique position to do that, as professors. We may as well learn how to do it.

John: And I’ve noticed that students are much more receptive to that than they were 20-30 years ago, where there appeared to be more of a stigma associated with that. So reaching out that way can make a big difference. And I know I’ve been referring more students for mental health assistance on campus than I ever have before.

Robert: And we can contribute to that continuing evolution by making our classrooms a safe place by saying “I love to go out into the gardens and meditate periodically, I experience stress and sometimes get physical symptoms. It’s a great way for me to cope with my stress.” By just saying that, I’ve signaled to students that it’s okay to talk about it. Sometimes two or three weeks before the end of the semester, I’ll take a meta moment and say, “Hey, some of you have been here longer than others, what are some keys to not exploding during the final two weeks of the semester. Go!” A discussion erupts in which they’re remarkably candid, and they’re spot on. They talk all about things that Bonnie included in our chapter on wellness. But again, it makes it safe, and they get some good counsel from each other.

Bonnie: I agree. And when we bring that to the classroom, we actually are real, and say, “You know what? I do have some stress. And I’m probably a little overwhelmed right now. And I want to back off and love you guys and set some boundaries for ourselves too.” Like, “I’m going to check my email within 24 hours. But usually after six o’clock at night, I’m done checking email…” and let them know that you’re gonna take care of yourself too. And hopefully, they can see that that’s important to you, and they will say “I need to take care of me too.” And it’s okay to have those conversations and as you develop those relationships, you can be a little more candid and they will feel, I think, more free to come to you when they do have a problem if you’re honest and authentic and say, “Yeah, I’m a real person, like I actually have to eat [LAUGHTER] and take care of myself and I have a goal this semester.” I told my students, on Wednesday, we started, that I’m going to do the lazy woman iron. Is it like the iron woman?”

Robert: Yeah, the Iron Man.

Bonnie: Yeah, the Iron Man, and I said, you know, I’m not good at it. I want to get back in shape after COVID and Christmas. And I said, if I see you guys at the gym, that’d be awesome. Please don’t make fun of me. [LAUGHTER] But I’m gonna be on the treadmill trying to get my miles in, and in a month, I get to do an Iron man. I think I can do that. And it’s just fun to open up with them. And there’s possibilities as you’re studying hard, you can still take care of yourself outside of that, and as your professor, I’m gonna take care of myself, because I want to be good for you. I want to be healthy for you. And I want to be excited to be here tomorrow and the next day, and to do that I want to take care of myself too.

Robert: So promoting wellness practices, I think we’re uniquely positioned to encourage students to get enough sleep, to eat well, to exercise, without being preachy about it, and we’re vulnerable in the way that Bonnie just described. That could inspire a number of students to step back and think maybe I could incorporate more exercise into my daily routine.

Rebecca: Again, those are small things that don’t take a lot of time. It’s a small little conversation or a small little anecdote that you share to set the stage for wellness. So it’s not as hard as sometimes we imagined it to be.

Bonnie: I don’t want to overwhelm professors either, ‘cause it could get overwhelming thinking I need to do all this, but, like you said, one or two things can make a big difference. Just a simple thing that Rob inspired me with and Steve too, to talk about. We live in a place where it’s cold, a lot of the year, [LAUGHTER]…

Rebecca: … me too.

Bonnie: … lots of snow here in Idaho. Yeah. And so it can get a little depressing anyway, because of the climate here. But, I send a roll around to see who’s there every time because I do keep rolls, not for the grade, but just because I want to know who’s there. And I say, “If you’re here I want to know, if you’re not here, I want to know. So I send the roll around, and I sent a question around yesterday about what are some fun things you can do outside in Rexburg, like when you need a break. And so then they write their name down and give ideas and so we send that around and it’s super easy. I didn’t have to take any time out of class for that. They signed up and then the other day I asked, “So, what’s your favorite comfort foods?” just kind of get to know them and show that you care about them. But it’s like a super easy way to take care of things and to inspire them to maybe do some wellness that week. [LAUGHTER] And think about those things that they might not be thinking about.

Robert: Rebecca, you have mentioned this a couple of times that it caused me to look up a quote by one of my mentors who happens to be the father of our current university, President Henry B. Eyring. He talked about how small changes can often have a big impact. And he said, “The best place to look for small changes we could make in things is in things we do often. There is power and steadiness and repetition.” And if we can lead by inspiration, or intuition, if you will, choose the right small things to change, consistent change will bring great improvement. So really, there are some things that we suggest, that if they were to change from one final at the law school to four different tests, and then a comprehensive final, that’s a bigger change. But much of what we advocate in the book is something that you can do quite simply, and much as I love many coherent systems of teaching, they intimidate me, like Project Based Learning, it sounds really cool, it’s just been a bridge too far for me. I just haven’t been willing to make the huge investment, it feels like I would need to make to switch my course over to that all the way. On the other hand, I could show up to class a little bit early and sit next to a student and get to know her, see how she’s doing, and connect with her. That’s a small change I can make.

Rebecca: Related to mental health, one of the things that many of our colleagues have certainly noted and there’s been many articles in The Chronicle and other places about this is the idea that students seem pretty disengaged right now. They’ve survived multiple semesters of COVID and other world complications and seem disengaged. Sometimes they’re doing the work. Sometimes we’re seeing students disappear. Sometimes they’re in class not doing anything. Sometimes they’re doing stuff outside of class. It looks different depending on the students. But there’s this general sentiment of disengagement. How do we help students feel engaged or reinvigorate their energy around learning?

Robert: Let me try three concrete ideas and then I think Bonnie might have some as well. First, at the risk of beating a dead horse, the more connected they feel with us, the less disengaged they tend to be in our classroom. I noticed when I sit next to a student and chat a little bit before class, that student who has not made a comment all semester, a good chunk of the time they will volunteer a comment for the first time. It seems to be a strong correlation there. Another thing that I learned in researching this book, and we did a survey of our students, one of the things that causes the the most anxiety was when the course was moving on, the class was moving on, and they felt like they didn’t understand something. I heard Sal Khan at a talk at Stanford say that the problem with the monolithic approach to higher ed or to education in the United States is that we kind of assume everybody moves at the same speed. So we did a test on chapter one, which is learning to ride a bike and a bunch of students get a C or a D or an F, and then we move on to riding a unicycle in chapter two, and we’re surprised when they fall off. We’ve given them no incentive to go back and master bike riding first. Just this week, I had a conversation with a student who’s a family friend, he grew up in our neighborhood, and he’d struggled. He said, “This is my redemption semester.” He was going to do some things differently. And I said, so here’s what to do differently, especially in your math class. When you get a poor score on a test, try to figure out what things you didn’t understand and go watch Khan Academy videos, or go to the TA or the tutoring lab, and figure out what those things are. I happen to be here on campus last night, and I saw him at about 8:30 when I left, and he had been watching Khan Academy videos, and said this was transformative. It really hadn’t crossed his mind before to fill in the gaps. But what happens I think is if that train leaves with students not on it, they then get disengaged, they’re lost. So if we can build into our course design ways and incentives for them to master what they don’t initially master, I think they’ll remain more engaged.

Bonnie: I agree with all that, actually and thanks for bringing in mathematics.[LAUGHTER]

Robert: Always given you a nod when I can.

Bonnie: I could always use some advice and some help there too. But I wanted to add to that… I guess, maybe emphasize… the importance of that connection, and choice and passion. sometimes we get a little dispassionate with our lives, or we’re going in a direction, it just seems that’s not really where we want to be. And as a teacher, it’s an opportunity to get to know kids, find out what their passions are, and maybe help to see some of that passion in your course. And I realize this might take a little bit more time, but sometimes it’s worth it. When we think about our projects and I find out the students majors and maybe a little bit about what professors they’re also working with, sometimes I can tailor a project to them, like I talked about before, and the students can come alive. At the same time. I think we need to be realistic too, some of these students might not really be disengaged, they might be overwhelmed. They have a job outside of your class. They have another 16 credits they’re taking. They may have other family and things like that that they’re working with. So it might not be about you [LAUGHTER] or your class, it might be about all these other things they’re dealing with and we could try to give good counsel as advisors and mentors and invite them not to overbook themselves. It’s not about how fast to get there but about the journey and as you go, you can try those things, but sometimes, they’re just overwhelmed and they can’t do your class too and do it well and everything else. So, sometimes I do ask my students do you really want to take my class this semester? [LAUGHTER] I’d be happy to see you next semester, but maybe you do need to cut out something because the disengagement might be actually something else.

Robert: In fact, I’ll throw that out too. I try to proactively by the second or third of the week of the semester, now reach out to my students who are falling behind in terms of their grades. Most of them are falling behind, not only in my course, but other courses. And so now I try to counsel them a bit more holistically, not just get my assignments done, it’s “So tell me a little bit about your approach. How’s it compared to what you were doing in high school?” Nobody’s really, especially if you’re a first-generation college student, nobody’s explained the rules that in high school, you could get by with very little homework. And in college, it’s flipped. You’re supposed to spend much more time doing homework. I talked to one student and asked him how much he was studying every day and he said an hour. He said it proudly. I said, “No, I mean, like, for all of your classes, and he opens up his calendar, he said, “No, I’ve got a study hour every day.” And I said, “Oh….oh, oh, oh, did you know it’s supposed to be two hours for every credit hour, like if you’re taking 14 credits, you should be spending 28 hours. Think of it like a job, you want to put in like 40 hours a week. This was an epiphany. I think he was a first-generation college student. Somehow nobody had made that clear to him, and he was failing in almost all of his class. So when we’re proactive, reach out to struggling students early, we often find that they’ve got other issues going on, or just haven’t figured out the rules of the game for college life and how to succeed. And that can cause anybody anxiety.

John: One of the things you suggest in your book is that people consider exploring QPR training. I know we have that on our campus, and we recommend that faculty participate in that. Could you talk just a little bit about that, and what its role may be in dealing with students who face more severe challenges.

Robert: So for me two big takeaways are that just as if you were playing soccer, and a friend crashed into somebody, and you could see the bone sticking out, you would say, “let me help you go to the emergency room.” You’re not a doctor. But if they say, “No, no, I’m fine.” You say “I see the bone sticking out. I haven’t been to medical school, but that seems like a bad thing.” Let’s get you into the car in to the doctor. So just knowing that it’s okay, I think sometimes we feel like it’s illegal for me to engage in counseling. Therefore, I can’t say anything at all about this. So that QPR is kind of a twist on CPR that just as if there are no doctors around and someone’s had a heart attack, it’s helpful to have a civilian do CPR, It’s helpful if I’ve got a student in my office, who I can tell as a lay person and with a little bit of training, wow, they’re struggling to get out of bed. And so now, the other thing I came away with from that is, it’s okay for me to ask “Are you feeling suicidal? Have you had thoughts of taking your life? Have you got a method?” Let me take you to the counseling center, and then just kind of spot. And then the other big takeaway for me was that studies show if they will promise you that they won’t take their life without calling you, people are much less likely to take their life. So I wouldn’t have felt comfortable or thought that was appropriate before I took that QPR training. I found that it’s made me feel like a lay clinician, and it’s alright for me to talk about those things. And now, over the last three years or so, I’ve taken several students who are suicidal to that counseling center. In fact, in a church setting, I was talking about this, and a young woman who was a leader in her congregation texted me that Sunday night and said, “Would you walk with me tomorrow?” She was a leader. She knew exactly where the health center was. She didn’t need me to show her. But she wanted someone just to walk her over there. And I asked her before we went, “Are you feeling suicidal?” And she was. So just to be that friend who can connect people in dire need with mental health professionals is a critical role that I think any of us can play in any walk of life, but especially as teachers.

Rebecca: For those that aren’t familiar, QPR stands for Question, Persuade and Refer.

Robert: Thank you.

Rebecca: I was looking enough just to make sure I had it right. [LAUGHTER]

John: And the training tells you what questions to ask including, “Are you feeling suicidal?” and then persuade them to get assistance and refer them to the assistance, including walking people over when needed.

Rebecca: I know that when I went through the training, I quite literally used it the next day. So it’s a useful thing to take the time to learn. And it does give you the tool set or the toolbox to feel like you can engage when necessary.

Robert: It probably gets your antennae up too.

Rebecca: Yeah.

Robert: You might have spotted that thing before but I think I’m more likely to spot some things now, to see disengagement that’s kind of come on quickly to the student in class or absences and reach out.

Rebecca: Or maybe to act on the thing that you spot that you weren’t quite sure of.

Robert: Yes.

John: We always end with the question: “What’s next?”

Bonnie: So next, I’m going home. [LAUGHTER] I can answer this so well, and eating the dinner that my husband, I think, made for me [LAUGHTER] after this first stressful week of the first week of school because he knows I’ve been so intentional this week about trying to be there for my students. I’ve worked long hours this week. I haven’t taken care of myself as much as I want to. So I’m going to go home, take care of myself this weekend, and hopefully, on Monday be refreshed for my students and I’m so excited to keep using some of these ideas and becoming their friends. I love working on my classes because I feel like we’re friends, and it’s only been three days, but I can’t wait to see my friends on Monday and I hope they feel that teamwork and that team that we become, I think, as we become a team throughout the semester. Some of these things don’t come up as often in my class, I don’t see the stress as often as they feel that teamwork. So I think the next thing for me is just to continue to work on being a good team member and creating this team that I want to see this semester… after I get some good dinner and a good night’s rest. [LAUGHTER]

Robert: I’m glad you at least include a good dinner because I feel like my answer is very selfish [LAUGHTER] compared to your selfless outward facing answer. But my career’s been unusual. I’ve spent about half of my time at BYU Idaho in academic leadership. And so just in the last couple of years have been able to dive in and research and write about these things. I’d kind of like to go speak to people who are interested in hearing about this, do workshops together with people who are learning how to improve this. I also helped teach a course that I designed and have team taught for the last six years with a couple of colleagues here for our new faculty. It’s a semester-long course. And so I’ve recently written a textbook for that course. That’s kind of all the stuff we think new teachers should know. I think we’re calling it Architects of Learning. And so I’ve got that to a point where eventually I’ll pursue publishers, figure out how to get those ideas out there as well. But build on some of the kind of the same stuff that’s laced through this book about just being intentional about the decisions we make in our tactical classroom decisions and our course design decisions can go a long way to improving learning.

Rebecca: Thank you so much for joining us. We really appreciated getting to know you and your work better and sharing it with our audience.

John: While we were able to see an advance copy of this, we are wondering when the copy will be released to the public. Mine is on preorder.

Robert: Yes, so it’s preorder and it’s March or April, West Virginia University Press has told us. So we’re looking forward to that actually getting out there soon. And we’re so grateful that you would have us on your show. We love connecting with kindred spirits who care about teaching and learning and education can do to make people’s lives better. Thank you.

Bonnie: Thank you.

Rebecca: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

268. Advancing Inclusivity while Mitigating Burnout

This episode is a live recording of a panel session at the Online Learning Consortium’s Accelerate Conference in Orlando on November 17, 2022. The panelists were Michelle Miller, Liz Norell, and Kelvin Thompson.

Michelle is a professor of psychological sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. She is the author of Minds Online: Teaching Effectively with Technology and also more recently, Remembering and Forgetting in the Age of Technology: Teaching and Learning and the Science of Memory in a Wired World, which was recently released by West Virginia University Press. Liz is a political scientist, and an associate professor at Chattanooga State Community College. She is also an experienced registered yoga teacher with over 500 hours of training completed. She is currently working on a book on Why Presence Matters in High Quality Learner-Centered Equitable Learning Spaces. Kelvin is the Executive Director of the University of Central Florida’s Center for Distributed Learning, and graduate faculty scholar in UCF’s College of Education and Human Performance. He developed the open courseware BlendKit course that many of us have taken, and cohosts TOPcast, the Teaching Online Podcast.

Show Notes

  • Panelists:
  • Miller, M. D. (2014). Minds Online: Teaching Effectively with Technology. Harvard University Press.
  • Miller, M. D. (2022). Remembering and Forgetting in the Age of Technology: Teaching, Learning, and the Science of Memory in a Wired World.
  • Norell, Liz (forthcoming). Why Presence Matters in High Quality Learner-Centered Equitable Learning Spaces. West Virginia University Press.
  • BlendKit
  • TOPcast
  • Brandon Bayne’s twitter post on the adjusted syllabus.
  • Nick Sousanis’ syllabi
  • Behling, K. T., & Tobin, T. J. (2018). Reach Everyone, Teach Everyone: Universal Design for Learning in Higher Education. West Virginia University Press.
  • Teaching Online Pedagogical Repository (TOPR)
  • Joosten, T., Harness, L., Poulin, R., Davis, V., & Baker, M. (2021). Research Review: Educational Technologies and Their Impact on Student Success for Racial and Ethnic Groups of Interest. WICHE Cooperative for Educational Technologies (WCET).
  • Hammond, Z. (2014). Culturally responsive teaching and the brain: Promoting authentic engagement and rigor among culturally and linguistically diverse students. Corwin Press.
  • Eaton, Robert; Steven V. Hunsaker, and Bonnie Moon (forthcoming, 2023) Improving Learning and Mental Health in the College Classroom. West Virginia University Press.
  • Enneagram test

Transcript

John: This is a recording of a live panel session that took place during the Online Learning Consortium’s Accelerate Conference in Orlando on November 17, 2022. We hope that you enjoy it as much as we did.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Welcome to “Advancing Inclusivity while Mitigating Burnout.” I’m John Kane, an economist and the Director of the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

Rebecca: I’m Rebecca Mushtare, a designer and an Associate Dean of Graduate Studies at SUNY Oswego. We’re the hosts of the Tea for Teaching podcast and the moderators for today’s session. We’d like to let everyone know that we’re recording this session in order to release it in an upcoming episode of Tea for Teaching. And today’s session will include 30 minutes of moderated discussion, followed by 10 minutes of questions from all of you, and I’ll turn it over to John to introduce our panel.

John: Our panelists are Michelle Miller. Michelle is a professor of psychological sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. Michelle is the author of Minds Online: Teaching Effectively with Technology and also more recently, Remembering and Forgetting in the Age of Technology: Teaching and Learning and the Science of Memory, which was recently released by West Virginia University Press. We also have Liz Norell. Liz is a political scientist, and an associate professor at Chattanooga State Community College. She is also an experienced registered yoga teacher with over 500 hours of training completed. She is currently working on a book on Why Presence Matters in High Quality Learner-Centered Equitable Learning Spaces. We also have Kelvin Thompson. Kelvin is the Executive Director of the University of Central Florida’s Center for Distributed Learning, and graduate faculty scholar in UCF’s College of Education and Human Performance. He developed the open courseware BlendKit course that many of us have taken, and cohosts TOPcast, the teaching online podcast.

Rebecca: This wouldn’t be a complete episode of tea for teaching if we didn’t ask about tea. So Kelvin, our coffee drinker, are you drinking tea today?

Kelvin: I am drinking tea today, Rebecca, this is a mint tea from Tazo. I have to admit, I went looking for my favorite conference tea, which is that orange Tazo tea that I looked everywhere and they didn’t have any.

Rebecca: Ah, a little bummer. Michelle?

Michelle: Well, true to form, I’m drinking coffee and I’m going to admit that I also mixed it with Swiss Miss hot chocolate, [LAUGHTER] a very guilty pleasure on many levels.

Liz: I don’t drink anything hot. So I’m drinking diet coke.

Rebecca: Woo!

Liz: Thank you.

Rebecca: As many guests before you have as well.

Liz: Yeah. [LAUGHTER]

Rebecca: John?

John: And I am drinking an English breakfast tea today.

Rebecca: And I have some very strong Awake tea, so that we’re really awake for this episode.

John: The pandemic spurred a rapid transition of instructional methods and an increased focus on inclusive teaching, and we want to talk a bit about that in today’s session. We’d like to ask the panelists: What were some of the improvements in inclusivity and building a sense of belonging, that you’ve observed during the pandemic? …with particular emphasis on remote and asynchronous courses. And Liz is going to start.

Liz: What I found, particularly in asynchronous online courses, is that I have noticed this has much broadened the ability of neurodivergent students and students who have some kind of disability, especially an invisible disability. And the way that it’s done that is by really forcing us to accept multiple kinds of student engagement. So what I noticed in March of 2020, is that students who had been almost invisible in my face-to-face class, when we came back to Zoom after spring break, I didn’t see their faces, but I heard their voices, if not vocally than in the chat. And I have so thoroughly appreciated that. I also learned from my students that you can change the way you show up in Zoom by getting one of those little lens things over your camera and changing your virtual background. So oftentimes, they would come into a Zoom room, and this would be in a synchronous class, and they would have a different cartoon character of the day. And this is a wonderful way to get to know more about them and their personalities. What I’ve also found is that when we are meeting synchronously, recording the classes is really helpful for a lot of learners. And so, since I’ve gone back to face-to-face teaching, I always open a Zoom room and record it and sometimes students will join by Zoom because they can’t come to campus that day. My partner who teaches math has also started doing this because they found that it’s really helpful for the students to be able to review what they did in class when they’re trying to apply it in their homework or preparing for an exam. The last thing I want to mention is that I teach political science, we always do a unit on civil liberties. And one of the things that I did in person was bring in a prop box and have them do skits where they acted out the precipitating events to major Supreme Court civil liberties cases, and I thought there’s no way we can do this over zoom, right? Yes. And it’s just getting really creative with Zoom backgrounds, with props that they have in their houses. It makes it so much more campy that it’s much more fun, because in person they always like “Oooh, I’m nervous.” In Zoom, it’s going to be a train wreck, okay, we’re just gonna embrace that, and that really creates a really fun dynamic in the classroom.

Kelvin: I would agree with the broad brushstrokes of what Liz said, I would like to broaden it further though, and say that throughout the last couple of years, what I’ve seen in a way that I’ve never seen before is an emphasis on empathy, and what we might call a human first orientation. I was really heartened to see that in a lot of ways in the early days of the lockdown era of remote instruction, and I still see ripples of it now. Two things that come to mind from the early days that are worth a review, if you haven’t looked at him in a while. One that got kind of viral, the so-called adjusted syllabus of Brandon Bayne from UNC, where he got really kind of real and transparent with students and released it out all over for adaptation. And then Nick Sousanis from San Francisco State University more recently had what he called notes on now as a syllabus section that just kind of, in a more encapsulated way, he sort of said that he wants to keep anchoring down to that human-first element, and just keep that a present part of every course’s syllabus.

Rebecca: Thank you, both Liz and Kelvin for giving us some insights into what we’ve already been experiencing and reminding us about some of the strong improvements that we’ve had. Many campuses are moving back to onsite instruction, while also considering how much of their portfolio should be including online options. And while each campus continues to adapt and adjust, how can we maintain the momentum around inclusive practices and avoid slipping back into our old habits? Michelle, will you start us off?

Michelle: And I love the framing here as momentum, because I think that many campuses have generated a lot of great energy around the why of inclusion, the why of equity, the why of justice, and why this is so important. The trick is going to be to follow that energy on with more on the what and how… how do we put that into practice. And I really struggle when I think about it, because our students, they need practice when there’s a new skill, a new way of responding. They need practice in lots of new contexts. And I talk a lot about transfer. And we may remember a skill or develop a skill in one context and we just don’t remember to activate it and carry it through in a new context. And we’re going to need that. We’re going to need to have certain techniques and teaching moves that do become second nature to us and be able to recognize, hey, this is one of those opportunities to put those into practice. So I was just doing a session this morning by a group from the University of Wisconsin. And they talked about, for example, the need to provide lots of concrete examples to faculty like what does this look like? Maybe even in your discipline? So, for example, increased course structure, we know this is proven to be an inclusive teaching practice. So I need to say, “Oh, yeah, this is the place to do this.” The frequent low-stakes assessments, we know that that reduces disparities in opportunity and disparities in achievement across many student groups. And so offering those concrete paths, making sure we know how to set them up, and then showing faculty that we see it when they’re making the effort and rewarding that. We don’t always need that little extrinsic reward, but institutions have to really keep that up if they’re going to see that this carries through, because that’s going to be the difference between institutions that really have that momentum moving forward and the ones that fall behind.

Liz: I agree… [LAUGHTER] with what she said. I come at this from a little bit of a different perspective, because I am in the classroom, or at least in a virtual classroom. But I think the word that comes to mind for me when I think about momentum is intentionality. And I think if we can remember to teach through our values, especially those values that became so crystallized during COVID, that’s a really good first step. And I’m fond of saying that what’s good for students is good for us. So I want us to think about the ways that we prioritized care, humanity, grace with our students, and also make sure we’re doing that for one another and for ourselves. And I mentioned this before, but COVID really revealed a lot of inequities that had been there for a long time. And now that we know that they’re there, we have to be very intentional about ensuring that they don’t just kind of go back to being invisible again now that we’re not all on Zoom together. So those are the things that I’m thinking about. It’s really leaning into our values, not forgetting what we learned, being intentional and doing for ourselves what we have seen we need to do for our students.

Rebecca: That’s a great reminder to think about not just the students, but the whole community, faculty, staff, etc.

John: During the early stages of the pandemic, we saw a lot of experimentation. And we saw the rapid growth of remote synchronous instruction and HyFlex instruction. And one of the things I think many of us are wondering is what roles they will play as we move forward into the future. Helvin?

Kelvin: Well, my crystal ball never was working. [LAUGHTER] It’s not even broken. But here’s kind of what I think is happening and what may continue to happen. I think synchronous online options will continue to be a factor in our digital teaching and learning in a way that it was not before the pandemic. And it is certainly possible that what I refer to as true HyFlex may still have a place as well. It’s just, gosh, it’s hard to pull real true HyFlex off, right? It’s such a design-intensive approach. What I contrastingly refer to as pseudo HyFlex… dual mode, simulcasting, webcasting of classroom experience… that has challenges. It has a place, perhaps, in individual instructor preference. But I’m going to tell you, firsthand from faculty colleagues I talked to as well as data I’ve seen, it’s exhausting. And it’s challenging for students, the what is sometimes touted as student flexibility ends up just becoming sort of a lowering of expectations and come and go as you wish and the intentionality and momentum seems to dissipate. I’m a big fan of intentional design and modalities of choice, and so forth,,,that way, instead. So, I think synchronous is around and I think our mutated modalities are around. And true. HyFlex has a place. But I’d love to see a diminishing of pseudo HyFlex.

Michelle: This is sparking so many of my own thoughts that I’m really feeling in a way validated here. And I’ll echo a lot of this as well. I think that again, in true HyFlex, where I am fully engaging the folks who are all spread out, I’m fully engaging whoever’s in my classroom, and we’re achieving all of our objectives. That really is a lot. And it’s not the technology. It’s the cognitive capacity. And as a cognitive psychologist, I say, “Well, when you practice remembering to read the chat and read the room at the same time, those demands get a little bit less, but they don’t go away. It’s just too complex.” So I think that we are going to have, yeah, the true bonafide HyFlex courses going forward, they need to be designated, they need to be supported in a particular way. So they may not be unicorns exactly, but I kind of see that coming forward. What I feel like I’ve kind of settled into a more HyFlex light, HyFlex infused is, for example, if you’re home and you’re remote, you need to be remote today, I don’t have an elaborate Google doc with a structured discussion maybe for you. But you can follow along, you can send in a participation card, I do give a lot of credit for participation, and so we’re kind of moving to that. I do think that we still have a sense of possibility. I mean, we have definitely gotten those skills, and we will find new ways to use them. But even just the basics, like “Hey, you can record your class meetings.” And I offer that too as an option for catching up sometimes where needed. I really thought that was going to be a disaster for a lot of different reasons going on. I said that’s not going to work. And I will eat those words now. So we have a few of these tools in our toolkit that we can use in this lower key way, I think.

Rebecca: So Kelvin, you mentioned the ability of changing modalities and offering some flexibility here. And the ability for students to choose the modality that they take the course in is just one of the ways that we’ve seen increased flexibility. But we’ve also seen increased flexibility around attendance, assessments, and many other aspects of teaching. How can we continue to support increased flexibility for students without overburdening faculty. I’m really hoping you all have some magic that you can help with.

Michelle: Yeah, so increased flexibility without the overburdening of faculty… When I reflect on this, I think we can borrow from the plus one strategy that you might be familiar with from Universal Design for Learning. Tom Tobin and his colleagues who write about this… the idea being that, yeah, we’re working towards perhaps a very rich environment with many different ways to demonstrate what you know, to take a test, to participate in in a learning activity, but you don’t do it all at once. And I also think, too, that I know that I’ve gotten, I think, pretty adept at finding those opportunities for flexibility that don’t necessarily add more for me to do and I think an example of this would be the way I give exams these days. So during the pandemic, I said, there’s always going to be an option instead of sitting for a traditional test. So I have an option that’s an essay paper with length and scope parameters and so on. But students can structure this. I suggest they often structure it as an email home to your family about what you’ve learned during this portion of the course. I’ve had a few students who have run with it as a science fiction writing exercise, which works great in psychology. So people will report home to their alien commander or home base about what they’ve learned about humans on Planet Terra. And those are, if anything, just a change of pace for me to read. I’m reading the exams anyway. So I’m always about the efficiency and so that’s something that I’ve tried to capture as well. But here too, we have to remember there are risks when we offer these and faculty who do go out on that limb and try it the first semester or two… maybe it doesn’t work perfectly… they’ve got to be affirmed, and they’ve got to be protected especially untenured. Folks,

Kelvin: I feel like I want to report to my alien commander now. [LAUGHTER] “Mork calling Orson, Mork calling Orson,” that’s a deep cut call back, you have to be of a certain age to appreciate that. I agree with a lot of that. I don’t think I disagree with any of that. But I will broaden out a little bit and maybe anchor back, Rebecca, to your prompt a little bit. I personally think we should lean into intentional course design in intentionally designed and created modalities for which student flexibility is the intent. Asynchronous online is probably the most flexible thing we offer outside of maybe true adaptive learning, which is… don’t get me started… that’s another whole tough banana to peel. I need a better metaphor… bananas aren’t that tough to peel… [LAUGHTER] a tough onion to peel, there’s lots of tears. I don’t know… something. And then I think we need to reiterate to ourselves and to our faculty colleagues that there are codified effective practices in each of these course modality domains, like… a shameless plug… at UCF, we host the teaching online pedagogical repository (TOPR), an online compendium of online and blended teaching and design practices. So there’s stuff that we know that are research based, time honored, that work well and benefit students, things like the broad category of learner choice, which I think just echoes a little bit of what Michelle was saying. We don’t have to reinvent the wheel if we lean into those things. And there’s real student benefit for doing that.

John: The disruptions we experienced in education during the pandemic when people were learning remotely affected all students, but disproportionately affected students who are first-gen, students from low income households and minoritized students. How does higher education need to adjust to meet the diverse needs of all of our students, some of which were not as affected by the pandemic if they were in well-resourced school districts and others that were left behind a bit when they didn’t have access to the technology or the resources that better funded school districts had?

Kelvin: Yeah, we could do better. Even here at the conference in which we’re participating here as we record this, we saw even today, student voice represented on stage and I think we can do better with that. There’s knowing who our students are, there are data that we are collecting and could collect that tell us something. But there’s also just attending to students’ personal stories, keeping a human face on the student experience, being grounded in our work there. But broadening further, again, I’m a big advocate of research-based and time-honored teaching practices. And in our digital teaching and learning domain, many of those have been shown to benefit minoritized and disadvantaged student sub-populations. I would do a shout out here to Tonya Joosten’s research review that has its long title. It’s got educational technologies and student success and racial and ethnic groups of interest all in the title. It can be Googled, it’s free, it’s available, but she does a nice job of reviewing and synthesizing the extant literature and making some recommendations from that. And so there are ways in which our online and blended in particular design and teaching practices benefit everybody but in particular, benefit first-gen, minoritized students, and so forth. And here’s a little free factoid… in a little fascinating recommendation. She says, you know, there’s some evidence to suggest that reduced seat time blended courses at the lower-division level particularly benefit disadvantaged students subpopulations and then going into online courses at the upper division in the major kind of courses. Those two things together. You know, it’s stunning, that’s not whiz bang, whirly gig shiny, right? …but it’s just doing the work but as we align that it really has benefits that evidence would bear out.

Liz: I just want to start by echoing something that Kelvin mentioned, which is that if we’re going to serve the students we have and the students that we will have, we have to know who they are. And there are studies of generations and qualities, but I don’t think anything replaces asking the people in your classroom about themselves in a way that is safe and non-threatening, but comes from an authentic place of curiosity. So I do this with my students. At the start of every semester, I ask them questions, none of them are required, but I ask them questions about things like, what kinds of technology do you have? And how many hours a week are you working? And which of this very long list of challenges do you think you might face during the semester? What kinds of things can I do to help you? …just so that I have a sense of who’s in the room, and what might be going on with them. So just yesterday, I was teaching my Zoom class from upstairs in my hotel room, which I was very grateful for, because I’m an introvert, and this conference is overwhelming. So I was happy to have an excuse to get away for three hours to teach. But we spent 20 minutes just asking, “How are you? No, no, how are you?” Not, I’m fine. I’m okay, but what’s going on? And one of my students said, I’ve always struggled with depression, and I thought I had it under control. But the last week has taught me that I don’t. And it’s only because we’re in week six of a seven week class that that student felt like he could say that, and the whole class could say, “Me too.” And those moments, I think that is so critical. And it’s just the humanization that we’ve already talked about, and meeting people where they are, I want to put in a plug for Zaretta Hammond’s book about, I don’t remember the full title, but it’s the brain and culturally relevant responsive teaching. It’s got a lot of really good buzzwords in it. But it talks about how, when the brain is in a state of trauma, or stress, you can’t learn. And so we can talk about all of these things that we can do. But if our students are not in a place to hear it, nothing that we do is going to make a difference. And I think if we’re talking about how to teach the students we have, we have to be aware of that before we can move forward.

Rebecca: So you’ve nicely connected to our next topic. [LAUGHTER]

Liz: …almost like I meant to.

Rebecca: lI know, it’s almost like you knew it was coming. Campuses are seeing a rise in mental health needs in their student bodies, both prior to the pandemic and it obviously continues to increase. And shifting to more inclusive practices is shifting the relationship that students are having with the faculty, just like you mentioned, being able to be more open about some of these things. In my own experience, I’ve seen an increase in disclosure of mental health challenges and distress and the need to refer and actively engage in suicide prevention. These are things that I didn’t need to do five years ago, [LAUGHTER] or maybe I didn’t need to do but didn’t see, but definitely see now. So how can faculty and institutions more effectively respond to mental health challenges facing our learning communities? And obviously, Liz, you kind of started us off on some ideas here. Michelle?

Michelle: Yeah, I’d like to put another book on the radar actually, as well. And full disclosure, I was an editor on this book, but I really believe in it. It’s coming out next year, actually, it’s called Learning and Student Mental Health in the College Classroom, lead authors, Robert Eaton. And it is in this space of friendly, actionable strategies when we do have increased disclosure. I mean, I found too, that Zoom itself is a sort of a dis-inhibiting space, like many online spaces are and going in I should have known that. it’s like on paper, I’ve read it, but then it became very, very real. So what do we do with that? Because the thing for faculty is, we mean this well, but our minds immediately go to: “How do I fix it? Oh, my gosh, do I have to kind of do amateur therapy that I’m not qualified to do?” And no, that is absolutely not what we need to do next. It’s a good thing to be concerned about not crossing boundaries at that same time as we are responding to what students are disclosing for us. So the strategies that I like are those that say, well, let’s start by not accentuating the crises and the stresses that are there. And a few other kind of concrete things… For me, not asking for doctor’s notes. Boy, when I started teaching a long time ago, I was all about the documentation and “show me exactly what happened at the emergency room,” and that type of thing. So now I offer flexibility and many paths to catch up that much like universal design for learning are built in for everyone from day one of the course. So if there’s something that prevented you from coming to class or making that deadline, okay, we can talk about it if you want, but we don’t have to. And you can simply take those paths that I’ve already laid out for you. I think taking the time to empathize, many of us did practice those skills and we can continue to do so. When I get to talk to people who are just starting out as teachers, I say, “You have this big loud microphone, this megaphone to a student’s ear. Your words land very, very powerfully. So you really need to think about those.” I find that my students seem to come to me expecting very harsh treatment and very harsh reactions, when they tell me about what they’re going through. And so I remind myself to simply start with warmth and compassion and human empathy, and a way forward. So that’s something that we can do. And when all else may fail, I say “How would I want my own child, in the same situation, a crisis, which is hopefully temporary and passing… how would I want them to be spoken to? How would I want that email to be phrased? What kind of pass forward would I want for them?” So that’s how I take it.

John: We’re a little behind schedule. So we’re going to ask anyone in the audience to come up to the microphone. One of the issues we haven’t quite gotten to is the issue of preventing burnout. Maybe if Liz could address that briefly while people come up.

Liz: So I want to talk about burnout in a very particular way. Because I think we often think of burnout as a me problem. And I think of burnout as an every one problem. It is a system that is creating it. And so I want to tell every one of you listening, that you are not the problem, and you haven’t done anything wrong. But in order to mitigate burnout, we have to be aware of what our boundaries are. And that often requires getting really quiet, and listening to ourselves. Because we’ve been steeped in this culture that tells us that hustling and productivity is the only way you’re valuable. And that is not true. But you’re not going to be able to figure out what your boundaries are if you’re listening to those voices. So you got to step out of that for a minute, and figure out what really matters to you. And here’s a good way to do it. Think about yourself 20 years from now. What are the things that are going to have mattered about what you’re doing right now? And what are the things that you won’t really care about? Choose to spend your time on the things that will matter to you 20 years from now, and all of these other kind of petty political debates in your institution… because we all have them… that’s not what you’re going to be remembering when you look back on your career. Focus your energy and your care on the things that you’re going to care about over a career and make choices accordingly. I think that’s the advice I would give.

JELISA: My name is Jelisa Dallas, I’m representing the University of Phoenix. My role is as a recognized student organization manager, a program manager. And so my question was, “How do we effectively work with students in a way that’s going to help streamline their purpose as it pertains to connecting with them in the classroom? So a lot of students have a lot of things going on, and how do we streamline that to get them to stay focused and prioritize when so much is going on and around?”

Liz: Thank you for the question. I mean, I think it’s making sure that what we are doing, we’re transparent about why it matters to them. And it’s creating relevance to whatever their goals are. And of course, you can’t do that if you don’t know what they are. So again, you need to know your students, but designing the work for your class, so that it feels intrinsically valuable, I think, is the best advice I can give.

ALI: Hi, I’m Ali Kirwen, I am a course operations specialist with the University of Michigan, we are an all online program, or I work for an online master’s program. It’s asynchronous, we have students everywhere in all different time zones. And we have faculty that care a whole, whole lot about our students. So I’m wondering, as a designer, how can I help encourage my faculty to set boundaries with their students in a way that honors their joy of teaching and interacting with their students, but is working towards preserving them and not burning them out as well? [LAUGHTER]

Michelle: Well, I think that even subtle things like wording choice. So are there things that say over and above, do we give awards for the person who did the most? Or do we give awards for people who did the best? So I think that faculty, especially when they’re new, are really looking for those cues. And so those can be powerful.

Kelvin: What Michelle said… [LAUGHTER] No, what Michelle said but just maybe that underscore and amplify that for a second, I’ve thought about doing this myself, and I just haven’t. But I love the colleagues and maybe some of you are them who have those little statements in the bottom of your email, right? Like, “if you’re getting this email, when you’re not working, don’t feel some sort of obligation to respond right now. I sent it at a time that I was working, that might not be the time that you’re working. And, in a collegial sort of way, we can have that same kind of a vibe, I think, in our course interactions between faculty and students. And then again, wording… I got a note just last night on a book project I’m working on from a colleague who’s gonna peer review. And he said, “I show my respect by being direct and blatantly honest.” And so I think framing is everything. Word choice is everything. And making explicit where you’re coming from without just sort of caving to the implicit pressures and cultural expectations. That’s hard. But it’s important.

Liz: Yes. And women in the academy, we do a lot of emotional labor. And that comes very naturally to me, if you know anything about the Enneagram, I’m a 2, like, I want to fix all your problems, okay. But the way that I do this, because students tell me everything. I think they tell me everything, because they know that I will listen. But what I have learned for myself, and this has been the difference between moving towards burnout and not, is that I am here to listen to you deeply. I am not here to fix your problems. And I will refer you to people who can help you. But if you just need to be heard, I will listen. And sometimes that does get heavy. But I also recognize that if I fix all the problems for all the students, I’m going to be exhausted. And that doesn’t really serve them in the long run. Because they don’t need someone to fix their problems. They just need someone to tell them that they’re okay and that I believe in you, and I care about you. And so I would say that, but deep listening is one of the greatest gifts you can give someone else. And that’s just listening to understand what they’re saying, not to respond. So that’s what I would say. And please never send pictures from the ER, never. [LAUGHTER] You know, students tell them, no ER photos,

John: We always end by asking: “What’s next?” …and we’ll start with Kelvin.

Kelvin: Crystal ball, still not working. [LAUGHTER] But here’s, I guess, my read on the situation, there are more options for people to learn and work than there ever have been before. So I think that we can either meet the needs of our colleagues and our learners, or someone else will when they go elsewhere.

Michelle: What’s next for me, I want to keep doing the absolute best I can teaching my actual classes. I’m just reconnecting so much with that. And I think that’s a good thing for now. And I’m eagerly looking at, hopefully, deeper changes in the academy, also looking for rebuilding online presence. Twitter used to be a big part of my online life and I quit that in May, and saying what kind of connections was this yielding with other faculty, even with students and the big ideas in my field. And maybe I can take more charge and have more agency in that going forward?

Liz: I’m working on a book. So you should all buy it when it comes out. Michelle is the editor, so it’s going to be really good. [LAUGHTER] It’s tentatively called The Present Professor. And we mentioned it in the introduction. But really, I’m just looking for ways where we can create more student centered,e students supportive and faculty well being centered places of learning. That is my passion. And I’d love to talk to any of you who are interested in that.

John: Thank you all for coming. And on your tables, there’s little packets that Rebecca designed with some tea in them, so please take them with you so she doesn’t have to bring them back to Oswego.

Rebecca: Yeah. Thank you all for joining us. And thanks for those that asked questions. And thank you to all of our panelists and we finished on time. [APPLAUSE]

Liz: It is exactly 12:30.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

266. The Secret Syllabus

Students transitioning from high school to college, especially first-generation college students, are thrust into a new environment for which they are often under-prepared. In this episode, Jay Phelan and Terry Burnham join us to discuss strategies that students can use to successfully navigate the hidden curriculum of college.

Jay is a biologist at UCLA and the author of What is Life? A Guide to Biology. Terry is a finance professor at Chapman University and the author of Mean Markets and Lizard Brains. They are the co-authors of the international bestseller Mean Genes. They have also recently published: The Secret Syllabus: A Guide to the Unwritten Rules of College Success.

Show Notes

  • Hertweck, K. L. (2014). Mean genes: From sex to money to food: Taming our primal instincts, by Terry Burnham and Jay Phelan.
  • Phelan, J., & Burnham, T. (2022). The Secret Syllabus. In The Secret Syllabus. Princeton University Press.
  • Charles Plott
  • Vernon Smith
  • Brickman, P., Coates, D., & Janoff-Bulman, R. (1978). Lottery winners and accident victims: Is happiness relative?. Journal of personality and social psychology, 36(8), 917.

Transcript

John: Students transitioning from high school to college, especially first-generation college students, are thrust into a new environment for which they are often under-prepared. In this episode, we discuss strategies that students can use to successfully navigate the hidden curriculum of college.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Jay Phelan and Terry Burnham. Jay is a biologist at UCLA and the author of What is Life? A Guide to Biology. Terry is a finance professor at Chapman University and the author of Mean Markets and Lizard Brains. They are the co-authors of the international bestseller Mean Genes. They have also recently published: The Secret Syllabus: A Guide to the Unwritten Rules of College Success. Welcome, Jay and Terry.

Terry: Hello.

Jay: Thank you. Good to be here.

John: Today’s teas are:… are either of you drinking tea?

Jay: I’ve never had tea in my life, I’m sad to say… or coffee. But in the spirit of caffeine, I’m having a Diet Coke.

Rebecca: A very popular choice on this podcast for sure. How about you, Terry?

Terry: British mother, a lot of tea in the house at times, but very little in my body.

Rebecca: [LAUGHTER] We’ve got tea deniers around here, John.

Jay: [LAUGHTER]

John: As usual.

Rebecca: I have a black tea blend today. It’s just called All India Tea.

John: That’s a new one. And I am drinking ginger peach black tea.

Rebecca: So we’ve invited you here today to talk about The Secret Syllabus. Can you tell us a little bit about how the book project came about?

Jay: Sure, I’ll start. I was thinking about this because earlier today I was looking at what I think of as maybe the origin document, and it’s from about 25 years ago, it was maybe 1997. It’s a document, it’s 136 pages long. And it’s this collection of emails and other student interactions that Terry and I had had over the years where students are sincerely trying to get stuff that they want. And they, more often than not, go about it in completely the wrong way. It’s either unprofessional, or maybe it’s rude, or maybe it’s just bizarre, or for whatever reason it’s not likely to get them the outcome that they want. And at first we’d laugh at them. But after a while you start thinking, “no, this is kind of tough for them.” It caused me to look back to the 1980s, where I had been a student. And I had been a very promising student coming in from high school. I had done well, and then I got to college, and I just was a terrible, terrible student. And I got bad outcomes. And it was all a big surprise to me, because it just was a different world than I had expected, and I was not equipped for it, sort of those two things together. Terry and I’ve talked about this and laughed about this a lot, but finally it came to a point where we said, “well, we should be doing something about this, we should at least try to help them.” And that was maybe how the ball started rolling, but it had a long, long gestation time.

Terry: Right, I would add to that… idea is to help students be more effective, when in this sort of promotional view… work half as much get better outcomes, and outcomes are not just grades, but grades are important. And as Jay noted, both of us came from great undergraduate schools and we both stumbled. We ended up getting masters from MIT and Yale respectively, then our PhDs from Harvard. And you might think that we had no troubles as undergraduates, and we did. So almost everybody has a problem adjusting to college, even if they end up doing fine. For me, I’m going to admit that I vandalized a lot of classrooms at the University of Michigan. So I sat in these classes with up to 400 people, I usually sat in the back. It looked to me like the professor was like a little puppet down there on stage, and I felt completely disconnected. And these old wooden desks that you would bring the arm down on, and I wrote on those in my days of discontent, “Frodo lives,” “Frodo lives.” I scratched it into the wood. I went back and taught there many years later, and there were still some of the desks [LAUGHTER] that said “Frodo lives” in there. And I took a semester off from school, and anyway, so almost nobody has a dip when they get to college, and it’s not for the fact they’re not smart enough. It’s because they don’t understand the world of college. And having noted, literally now thousands of students who have had the same problems that we had, and sometimes worse, thought we would write a book to help students.

John: So you came in with a lot of advantages and experienced trouble, but many of our students come in and don’t make it through. And that’s especially true for students from low-income school districts where they’ve had few resources and very little college prep in terms of the coursework they had or guidance in terms of how to be successful in college, and many of them are first-generation students. So we’ve got all these students who come in, they borrow a lot of money, and then they end up without a college degree and a huge burden of debt. Would a book such as yours help these students in learning to adjust to the new culture that they’re suddenly being thrust into?

Jay: Well, that is certainly the hope, I’ll tell you. I’m a first-generation student, and my parents are both very smart, and I think could have done college, but they didn’t. And when I grew up, we were extremely poor. I had never been out of Northern California. I’d never been on a plane when I applied to college, I had to apply just sight unseen, and all of that. And I think that was a factor maybe in why I struggled so much when I first got there was that I didn’t have a world of people around me that that was just part of their life, “Here’s what you do, oh,” or “here’s how you struggle,” or “oh, here’s the story about how I figured out how to take notes or I figured out what my major was going to be,” …any of those things. I had no idea that those obstacles even were laying ahead of me. I had succeeded in high school, but I had done that, I think, just through brute force method. I put in the hours studying, I didn’t know how to study, but I put them in, and I did well. I had very lame interpersonal skills with the teachers, but they were a little bit less lame than the other students at my school. Together that allowed me to get through and even to get the message, “Hey, you’re okay, you’ve got it figured out. College might be harder, but just use more of the same.” And I at least now I think coming from that perspective, it makes it a little bit easier for me to get inside my students’ heads and realize the struggles that they’re maybe going to have. Terry talks about sitting in the back of the classroom and vandalizing the desks. For me, I wish that I had gotten inside the classroom. But the idea that I was somewhere that nobody was checking up on me, I would walk to where the class was, and then I’d hang out outside, I talked to friends or I’d just sit out there, and it was this very empowering act that I was in charge. And that for me led to most of the problems that I wasn’t even in the classroom. And I had this idea then that the teacher was just going to smash me down to show me that they were in charge or something like that. It was all misguided, but I think a lot of that came from, as you say, John, that I had no background, this was completely foreign to me. And because of that, it’s hard enough certain obstacles to overcome them when you see them ahead. But when you don’t even know that they are in front of you, it gets really difficult.

Terry: The Secret Syllabus is a Princeton University Press book. So it went out for academic review. And there were a number of comments: “This book should be directed only at first-gen students, because that’s really where the problem is.” And our response was, “It’s probably more important for first-gen students if they’re able to understand it and we’ve written it in a way that they can use it.” So I have three daughters, one is a freshman in college, and the other two are, one’s in high school, one’s in middle school, and they get the knowledge from The Secret Syllabus directly from me. Now they don’t listen very often to that, but they at least have around them all the time both explicitly, like “how would you interact with your teacher on this topic?” And then in the air, in the gestalt of “how do I interact with the pediatrician?” and so forth. And so if you’re not in a household like that who knows how to navigate this world, where are you going to get the information? Well, one answer we hope, if we’ve done our job, is The Secret Syllabus. So even though it’s not explicitly targeted only for first-gen students, I think it actually has more use to a first gen student who doesn’t have access to a family member or a friend who knows the secret rules.

Jay: If I could add one thing, also the students, yes, who are not the first-gen students, or who have a world of experience college graduates around them, sometimes that sort of turns out to be a negative, because they might imagine that they’ve gotten the message “Oh, I don’t need this.” And in a lot of cases, it’s like, “Oh, yeah, Secret Syllabus. Yeah, something like that would be great for the remedial students.” And I think “what do you mean, the remedial students? You took when you were in high school, or even your first year of college, you took chemistry or history or English, or whatever, but who taught you how to study? Who taught you how to make a to-do list? Who taught you how to develop long-term career goals? Who taught you how to just plan your time?” all these different things, how to develop professional relationships, and a lot of these students who maybe have tons of advantages that causes them not even to realize that there is this whole secret syllabus of other stuff that they have no training in whatsoever. So they needed just as much.

Rebecca: I would actually recommend it as reading for faculty and administrators. Because when I was reading it, like yes, I actually knew these things. But like, I don’t typically explicitly state all the things because it’s the hidden syllabus [LAUGHTER] …these are all things that if we’re a part of the Academy, we have learned over time, but we don’t actually typically articulate necessarily or make explicit to others. And so reading this was like, “right, yes, of course, that is a rule that nobody knows about.” And so I could see a real advantage to having a faculty reading group, for example, around this book just to get insight into the missteps that students make simply because they’re not aware, and again around for administrators who may be in similar situations, like, thinking about policy, or easily blaming students, or easily blaming faculty for something, and just becoming more aware of all the unwritten rules around the academy seems really beneficial.

Terry: Amen.

Jay: That reminds me of a term that my wife, Julia, she’s an educational researcher, she knows all the official terms for everything, but expert blind spot; that you get so good at something that you don’t even realize that you know it or that you’re doing it. And it helps if you’re talking with other people, you can see it in them and at least call it out. So you’re right, you have a reading group, and you’re like, “oh, yeah, we do know that.” Or “we do have that whatever practice that we haven’t told people about.”

John: One of the things you describe early in the book is sitting in the back of the classroom, feeling completely disconnected from the experiences that the professor was describing. And you also talk a lot about how students come in for office hours and ask questions that faculty find really annoying when the students are really just trying to get some help and they don’t know how to phrase it. They’ll come in and ask questions like, “I really don’t understand the material we’ve been talking about for the last two weeks. Could you explain it to me?” [LAUGHTER]

John: And it’s really frustrating for us, but clearly the students are reaching out. And reading your book reminded me a little bit about how they must feel in these types of circumstances where they’re feeling a little bit awkward asking these questions. They really would like some help, but they don’t quite know how to phrase it. And I think part of our responsibility is to help bridge that gap a bit. And reading your book, I think, as Rebecca said, would be really helpful for that.

Jay: That’s funny, I always laugh when someone will say that, “Oh, I miss class. Can you just quickly tell me what happened?” I’m like, “If there was a quick way to do it, class would have been quick, then we would have been done.” [LAUGHTER] But you’re right, though, you have to immediately stop yourself from mocking them or looking down and think, “this person actually just came to me, they want some help.” So office hours is an unusual thing. And full disclosure, I only have experience with it as a faculty member, because the total number of times I went as a student: zero, and I regret that very much. But as I was going through college and struggling, so many people would even say, “well, you got to go to office hours,” I would think to myself, “okay, I hear you. I’ve heard lots of people say that.” But in my mind, I thought, “well, it’s okay. I know I have the textbook. I can read it or someone’s notes. So I’m okay.” Because I was making this assumption that office hours was for me to say, “could you re-explain everything about photosynthesis again?” Or “could you solve this problem?” Terry always makes me laugh about the student who comes into the office and says, “I need help with problem number three, and number seven, and 11 and 14 and 21 and 22. Can you redo those?” And they sit there and I think, “okay, yes, we do know the answer to those. And that is something that can be done in office hours.” But the insight… that it took me about 20 years to have… is that office hours is the first time, maybe the only time, where you as the student control the agenda. You are able to talk about anything that you think is relevant. And for that reason, yes, you can have them re-explain a concept, except that there’s Khan Academy, and there’s videos and there are textbooks, and there are other people and there might be a teaching assistant for the class, that might not be the best use of the time, because you have alternative ways. On the other hand, you’ve got this instructor here who has some insights that could really help you. And we try to give very concrete examples of things you as a student could ask. You could say some questions designed to help you be a better student… ask something like, “hey, if you were a student in this class, how would you spend your study time?” Or “you’ve had a lot of students, if you could change students in one way, what would it be?” Or if it gets close to the exam time, you could say, “to you, what are the elements of an ideal exam question?” Or “how do you create a lecture?” Or did you struggle ever? Did you always know that you wanted to be a biologist” or an economist or whatever, and you ask these questions, and in my office hours, which is usually a group of people, they be around and most students are like, “wow, this is really good and unexpected… the conversation we have helping them with professional development” or whatever. There are always a couple students who let out these heavy sighs, like, “ugh” because they have their list of 26 questions they want you to restate. And I have to usually directly address them and say, “I can do those. But this is actually some useful information that you’re not going to be able to get anywhere else, and might help you out, so think about that.” And I’ve even started now, in class, I’ll say, “hey, in my five years as an undergrad, I went to office hours zero times, you should just go even if you have nothing to say, just listen, just come and say hi, and you can leave” that I think has lowered the threshold a little and I get a bunch of students they come they say “hi,” they stay for a few minutes, and they realize, “oh yeah, this isn’t just a cram session” or something like that.

John: And once they come that first time, they’re much more likely to come up to you other times and ask questions in the hallway or just stop by for brief visits. And it makes it a much more pleasant interaction, I think, for everybody.

Jay: Yeah, they realize that you’re human, you’re a regular person, that you like to laugh, you have outside of class things that you can talk about. And that really does, I think, demystify who you are, and maybe what motivates you and other things that are relevant.

Rebecca: One of the things that’s really interesting, I think about this particular book is that we often deal out advice to students all the time that’s kind of generic and probably not that helpful. But when you really think, it’s like, “you need to manage your time better, you need to study more, you need to whatever the thing is,” without much explanation. Can you talk about some of the examples of maybe bad advice that students receive and how maybe you’ve addressed and provided a different kind of advice in this book?

Terry: Well, the office hours one is an obvious example. But we already talked about that. So it’s probably not the best example to continue on. I might say, I don’t know if it directly addresses what you said, but if the student and faculty member, or instructor, can have a relationship, professional relationship, as opposed to a transactional view, that’s a better solution. So I’ll give one cautionary story, which will make me sound really mean. And then I’ll tell you another story. So the mean story is I have a student come and he says, “I’ll talk about my grade on my presentation.” I’m like, “well, do you want to learn how to do a better presentation or do you want a better grade?” “Uh, kind of both.” So he just wants a better grade, but he’s kind of mealy mouthed about it. So he sits there for 11 minutes and goes through it. And then as he’s leaving, he asks, “oh, and is there any chance to get a job?” and I’ve gotten hundreds of people jobs, life-changing jobs, and I said “just took 11 minutes of my time, I could have gotten your job in those 11 minutes.” But he goes, “what about now?” I said, “no, you had your time. Get out.” So conversely, I had another student who used to just come by and chat with me. And we would talk about anything but the class. And I remember he bought me, this was years ago, an Austin Powers lifesize mannequin to put in my office with a frilly cravat. And then towards the end of the semester, another faculty called me and said, “Oh, you know, this person is in trouble in this stats class, what do you think?” and I said, “I think they’re an excellent person, that they are really trying hard. And I would give them the benefit of the doubt.” And so I don’t think explicitly, the student was doing anything to try to manipulate me, but because he came by and talked to me, and I got to know him a little bit. Interestingly, he’s running for Senate… in an important race. I won’t say his name, but he’s running for Senate next week, the same person, so his effective techniques for interacting with people…. So I don’t think bad advice people give you, it is just not advice that you get. And when you treat the person transactionally, then they treat you transactionally. And so if the faculty member is a fair and honest person, you’re going to get whatever you deserve as a student from them, they’re going to look at your grade, I’m going to talk to my student about his 8 out of 10 on the presentation, but you’re not going to get the next level relationship benefits, which can extend to the career and so forth if you go down that path.

Jay: Something that I’d like to add to that… when Terry and I first wrote our book, Mean Genes, people would read it, and they’d be all excited about it. And they’d come to us, and they’d say, “you guys should be on Oprah.” And that was their advice to us about “here’s what you should do. Because I know you’re trying to sell books, you should be on Oprah.” We’re like, “Oh, let me write that down.” And you see advice like that in books, oftentimes, where it’s well intentioned, and it’s actually accurate… we should have been on Oprah. But it wasn’t helpful, because it’s like someone saying you should go to office hours, they’ve already heard that. You have to tell them why should they go to office hours. What are the questions they should ask? Or someone says, “you should get a faculty mentor.” Okay, they know that, but how should they do that? And I think that oftentimes, you’ll also see in these books where they start doling out advice, that it’s just patently offensive, because it’s so obvious. Get to class early, be prepared, read the assigned material, try not to cram for tests, avoid procrastination. You might think then that alright, whatever, that’s harmless… that advice, but I feel like advice like that actually does extract a cost because you’ve only got so many words in a book and you’re either going to convince someone, maybe someone who’s reading Strunk and White, how to be a better writer, and it’s this very concise, precise book, so you get the message. But if the message is buried within stuff that the person thinks is obvious, they’re likely to put it down, or they’re likely not to trust you as much as the author because clearly, you don’t understand the reader if you think that they need to be told “avoid procrastination.” We all know that. That was a big part of our goal in writing the book. We didn’t want it to be a giant book filled with either obvious stuff or not helpful stuff. We wanted it to be a high density of material.

John: One of the things you talk about along these lines in terms of providing tips that may be different from the advice a student’s had is in terms of planning your pathway through college, because you note a lot of students are told they should pick a major early, focus on that, and work very quickly towards that goal as efficiently as possible and get their Gen Ed out of the way, and so forth. Your advice is a bit different. Could you talk a little bit about some of the advice that you recommend concerning what types of classes students should take and how they should plan their career in college?

Jay: As students are making their way from high school to college, we’ve seen this common pattern where someone will have, I don’t know, maybe when they were in fifth grade, thought, “Oh, what are my options when I grew up? I could be an astronaut or I could be a racecar driver or a pop star or professional athlete or a doctor. And if you happen to hit on Doctor, which is maybe the only realistic hope among those for most kids, your family responds better, like, “Oh, yes, very good.” And as the kid who it’s nice to have, it seemed like your family now loves you slightly more than they used to, that seems great. And then you start to stick with it, “Oh, yeah.” And then I get these students, they’re at college, and there may be a first year and they’ll say to me, “I’ve wanted to be a cardiothoracic surgeon since I was 10.” They want that to convey their commitment to it and their maturity and all that. But all I can think of is “do you really want to let a 10- year old make this most important decision of your professional life?” I think that maybe you should come to college with a more open mind, because when you were 10, you didn’t know all the options. If you look at these lists, and anytime there are published lists of job satisfaction, I love going through them because all the top things on job satisfaction, you’ve never even heard of as a 10-year old, or even as a college freshman, whether it’s software quality assurance engineer, or physical therapy director, or risk manager or security director, whatever. So from that, it’s caused us to realize that it shouldn’t be a race… who can identify their career goal first? …and then have them make other students feel bad because they’ve gotten to college and they can’t articulate it. It’s like, “oh, what do you want to do?” And the other person just talked about all these great things they want to do. And you’re like, “I don’t know, it sounds like you’re behind. It sounds like you’re worse off,” except that when you get to college, you don’t know what linguistics is, or applied math, or anthropology, or in many cases, psychology, or these majors that are huge, rich worlds that might interest you. So we want to encourage students to resist that temptation to have a plan. Because it seems so good, because everyone is so relieved and happy. But instead, you can’t just be passive, you have to explore the options that are open to you. That’s part of what you should be doing there. You have to learn about possible career paths, you have to investigate fields that might be of interest to you, you have to reflect on what is it that makes you happy? And what is it that stresses you out, maybe not in as good a way? …and that should be your plan because if you can’t explore things in college, then when do you get to explore? You stick to your 10-year old dream, and then you have the midlife crisis at 35 when it’s much harder to change your plan. And I think that message has not gotten out as much. Often students will even say to me when I say don’t worry about articulating a plan, but be active in exploring it. And I say, “you might have been experiencing parental pressure for a long time.” And they’re like, “oh, no, no, I haven’t at all. My parents are actually really chill about what I do.” I have to respond, “your parents are chill about what you do, because you’re still a biology major and that’s what you were when you came in. And you’ve articulated the same plan. The only way you’ll know if they’re chill is I want you to send them an email or call them today and say, “Hey, I’m changing my major to English,” then see if they are, because you may have been responding to subconscious signals about how much they love you when you have a plan versus not having a plan. And if we could get that message out that this is the time to learn about what you like and who you are, and that you might be becoming a different person than you were when you were younger.” That has a lot of value, I think.

Terry: So I invite my alumni to come back. I teach at Chapman now, and I have all these students who have graduated and have good jobs. And one semester, one of my students said, “I’m going to come back and I will talk to you students and I want to go to lunch with them. I want to interview them and I want to hire some of them.” So I tell my students “here it is, this great opportunity. The firm is local, if you want to live in Southern California. They pay well. The person who’s coming graduated four years ago. You want a connection with somebody who’s not remote from you? What could be better?” I said it’s only going to be room at the lunch for six people. So sign up and I’ll randomize publicly who gets the sixth slot.” So the person comes, Alan, and I go to do the randomization in front of all the students. There’s 70 students in the class. I look at the list, the sign up list, there are three people on the list. So all three of them go to lunch. And one of the kids, Mitchell, I just invited him back. I gave a talk to all the new Chapman students, all 2000 of them. And I invited him as the guest speaker to talk about this story. He comes. And I said, “why are you here?” And he goes, “I’m not even interested in real estate finance, but I just thought it was a great opportunity to learn something and come to lunch and talk to somebody.” He now works for that firm since that lunch. He got an interview, he got an internship, he loved it. And now he loves his job there. And it’s just hilarious that people miss this opportunity. And then again, showing my mean side, [LAUGHTER] a year later another kid writes me and he said, “I was in that same class. And now I want a job with that company. I’m interviewing with them. Can you connect me?” And I’m like, “no, you had your chance to connect.” I mean, of course, I didn’t say no, but the reality was, it’s much less effective to connect when you want something than when you’re just going to go learn something. So you have to look for those opportunities. In my own case, I was riding my bike through Harvard yard when I was in grad school on my way to a microeconomics review session. And I saw a poster for a talk by Caltech professor Charlie Plott. And I thought it looked really interesting. It was about experimental economics. I’d never heard of the field before. I’m like, “Okay, I don’t know what this is. I’m probably going to get a worse grade on my micro test, but I’m going to go to this talk by Charlie Plott.” And I went and was like, “Wow, I didn’t know any of this stuff.” And then I ended up doing my PhD with another experimental economist, Vernon Smith, and how I ended up at Chapman and I did get a worse grade on the test too.

Rebecca: You want all the things [LAUGHTER].

Jay: Can I add one other story that Terry has told me about one of his students who came in wanting to be a doctor, and was all excited that he was able to get this internship shadowing a doctor. And he goes, it was an orthopedic surgeon, and he gets to go, follow her around, even go into the surgery, theater, and all that. And what he discovered was, even though she loved her job, she was great at it, he was bored to tears. He thought, “Oh, my God, this would not be good at all.” And in some ways that felt like a failure, like, “oh, no, I don’t like what I’ve picked.” But in other ways, think about the cost of realizing that now when you’re in college and the cost of realizing that when you are in your fourth year of medical school or you are 10 years into your career. So the idea of exploring also carries with it that you’re not just trying to get the things to add onto your resume. You’re exploring because you’re going to find some things you don’t like, and it’s low cost to get to rule them out at the earlier age. So people don’t always realize that, that trying something out might give you information about what you don’t want to do. And that that has maybe as much value as finding out what you do want to do.

Terry: Right. So here’s something else that’s interesting. These people that we’re talking about are not made up things, this is a real story. So that person who Jay just talked about who was my student, who wanted to be a surgeon, here’s his most recent posts from LinkedIn, he now works as a product manager in London. He literally just posted this “when I was 10, I thought that my goal in life was to shoot for the stars. But today I understand my destiny is to come into the office today, every day for the rest of my life. Why do I say that? Because I came into the office this morning, and it was the best feeling in the world.”

Jay: That’s so great.

Terry: Isn’t it? I just looked them up while you were talking, Jay. And yeah, they’re all real people. And these are real stories. And we have hundreds of them that we draw from to synthesize the themes there. And that theme is very clear, which is, for most people, picking your job when you’re young and sticking to it is a bad decision. And if you’re talented, there’s so many different things you can do in the world. I had another friend, which was funny, he was a very talented guy too. And he said he realized when he was about 25, he didn’t want to be anyone’s boss. He didn’t want to have a boss. So he end up becoming a journalist, one you’ve seen on TV many times, but he was like, you don’t know those things when you’re 10. He didn’t know that at 10, he figured out when he was 25. Like, “this is gross. I hate giving performance reviews. This is terrible, even worse to give them than it is to get them.”

Rebecca: Another thing I really like about your book is these stories that you include, and that they are based in reality. And it’s a nice way to bring students into the conversation. And a lot of the stories have this thread of needing relationships [LAUGHTER] or a chapter on seeking out good teachers when you’re choosing your classes. And a lot of times those good teachers lead to good mentors in other things. And as you’re telling these stories, I’m thinking about the internship that I went on that I discovered what I didn’t want to do, or the mentor who told me I should take advantage of some of those opportunities that I probably wouldn’t have done had I been left to my own devices without a little bit of nudging.

Terry: So what was it that you didn’t want to do?

Rebecca: Well, interestingly enough, I didn’t want to be a graphic designer in the traditional sense. I do teach in a graphic design program. But what I discovered is I wanted to do more experience design and interaction design, which I hadn’t really been exposed to. I didn’t want to do marketing and branding, which is what the internship was. I hated it. I hated every second of it.

Terry: And when you changed your mind, was it difficult internally or externally to change what you want to do and have to tell everyone a different story?

Rebecca: Yeah, I mean, I think the first thing afterwards, it was like, “I’m quitting this major.” I actually ended up finishing it. But I ended up curating some of the other experiences around it in a different way. Because at first I thought I was gonna have a graphic design major, and I was gonna have a business minor. This is what many students have this combination of things. And I quickly discovered that wasn’t going to work for me and I needed to explore some other things. And I went more on an exploratory path for a while to figure out some other things, really in a field that was just starting to come into being.

Terry: Yeah, and did it feel bad, though? Now it sounds cool. Like, I went exploratory and it felt okay then?

Rebecca: No, it was really nerve racking and I was really stressed out and I tried to go to the counseling center and I talked to every mentor I had about how it was having a crisis. I mean, it felt really bad initially. [LAUGHTER]

Terry: We also tell three stories in The Secret Syllabus of people who found their passion early and stuck to it and then were very successful in an area, Steven Jobs among them. And the myth that they project is that you’re a loser if you don’t know what to do when you’re young. And they have this idealistic: “there’s one job for you, there’s one passion for you.” And that is not true for most people. And it’s a destructive myth out there. Most people are more like your experience, Rebecca, or my student, where you’re not sure. And if you’re introspective, you learn and you find something different.

Jay: I think, as an instructor, and as a parent, and just as a grown up in the world it’s very hard to internalize the message that Terry just said, which is that something inside of us when we meet a kid who says, “Oh, I’m applying for college,” or “I just started college,” we ask questions like, “Oh, what do you want to study?” These are questions that I think can subconsciously convey the message that if you have a plan, it’s slightly better than if you didn’t have a plan. So you have to really undo that and ask more about what kinds of things interest you or as you were saying, Rebecca, [LAUGHTER] when you describe something that was a bad experience, I find that if I ask people: “Tell me about your worst job experience,” or ask students “tell me about your worst classroom experience,” that can be more educational than hearing 100 stories about someone who saw the path illuminated and followed it and now enjoys success. If we can shape discussions in that direction more, it can really help a lot of students who I think don’t want to acknowledge that they’re not sure or they don’t have a plan.

Rebecca: I recently had a conversation with a colleague about a conversation she had with a student that she was mentoring. And the conversation went something like this, “I can’t wait to meet your parents at graduation.” And the student says, “my parents definitely are not going to be excited to meet you.” We were talking about this and how that really reveals how there’s a disconnect about the pressures that might be happening at home or from other sources that are happening to a student that aren’t always visible to us as faculty or mentors. There’s so many other things going on. So although you may have convinced the student of a particular thing or suggested they do this activity, or maybe you’ve helped them discover a new major or something that might suit their passions, they’re also having this internal dialogue with themselves about how am I going to break this to my parents or whatever needs to happen. But I just thought that conversation we had this week was so enlightening.

Jay: I love that because I was finishing my PhD, and after having struggled for years and years, and really things were touch and go all the way till the end, I’m getting my PhD at Harvard, and my parents come, and neither of them had been on a plane before I went to college. And they come now to Harvard. And I’m feeling all great about this. And just as I’ve gotten the diploma, my mom says to me, “Do you think you could get into medical school now?” …and she was still clinging to this idea of when I was maybe in grade school that I could be a doctor because then that would have some job security and respect or whatever. And so if I’m not a doctor, it must be because I couldn’t get in and maybe the Harvard PhD will help. And I had to tell her like, “I don’t want to be a doctor.”

Terry: And Jay, did you know that I got into medical school?

Jay: I did know that. I can only imagine the pressure because I know your dad. Yes. That’s just brutal. If my parents heard that I had gotten into medical school and didn’t go, the disappointment would have been overwhelming.

Terry: I had an excellent job as a busboy in one of the finest restaurants in La Jolla that I went to instead of going to medical school.

John: For many years, I taught in a program for gifted high school students at Duke University in the summers. And at the end of the term, we met with parents and many of the parents had very definite career plans for their 15- or 16-year old children. And they typically asked, “do you think our son or daughter has a future in,” I was teaching economics classes, “in economics?” And I’d say “clearly, they’re doing very well at this and it’s one of many things that I’m sure they could be very good at.” What I normally would say is, “over half of the students who enter college change their major within the first couple of years.” And what I’d recommend is that they explore as many different courses as they could now and when they go into college, and parents who didn’t generally like that advice very much. As you’re talking, I was thinking of two of my former students who I’ve known, now twenty-some years later, one of them actually was one of the best students I ever had in an economics class. Then he ended up at Stanford, he changed his major to religious studies and as he put it, “path of least resistance,” and he became a musician, and he’s a very, very successful musician today. Another student, his parents had plans for him for medical school, and he was on a Freakonomics podcast because he’s now an economist who’s become an expert in some interesting areas in economics. And they didn’t intend for him to go on in economics, he was clearly ending up in medical school.

Rebecca: I’m wondering if a good audience for your book is parents. [LAUGHTER]

Jay: Possibly not, I have sometimes thought about that. I might, if they saw me in class, wearing a nice shirt and tie and I looked very formal about the whole thing. I might be the most dangerous, subversive element in their life, because I’m the one telling their kid your parents already had the chance to live their life, you get to decide now. If they truly believe that you are wise and mature and smart, as they say they do, they should trust you to make good decisions. But it does feel like yeah, the parents… in the long run, they might… but in the short run, they might fear this kind of thing because parents want the best. And sometimes that means a little bit of risk aversion.

Terry: I would say I am a terrible parent. [LAUGHTER] So I have three daughters. Two of them are sort of introverted, and one is super extroverted. And I’m always nervous about the extroverted one, even though she’d probably be very good at navigating the world. But I asked her today, I was driving her to school today, and I asked her, “do you ever read your history textbook?” She said, “Absolutely not. It’s so boring.” I’m like, “yeah, it is boring. The content is good, but presentation is terrible.” Yes, so once you study hard, and don’t go out very much. I’m like, Yes, things are gonna work out okay for them. And then my middle daughter who’s doing well and super outgoing and loves to talk to people. I’m just nervous about the downside for her. So I see the attraction of neurology or being a was it, Jay, a…?

Jay: [LAUGHTER] A cardiothoracic surgeon? Yes, they don’t even know what that is. [LAUGHTER] I should validate. Yes, Terry is a terrible parent.

Terry: Yes.

Jay: And yes, his daughter is starting at Stanford this year. So that kind of terribleness.

Terry: One of the introverts?

Jay: Yes. [LAUGHTER]

Terry: The extrovert’s not going there, except to visit. [LAUGHTER]

John: You mentioned boring history books, one of the things that I really enjoyed about your book was how powerful the narratives are in it. And I think that’s something that perhaps we could all use when we’re writing books about our disciplines, or when we’re teaching these subjects, because storytelling seems to be really an important way of communicating information, and we don’t always use that in our classes.

Terry: It’s a failure of our ability to understand other people. So one of things I laugh about is, when I have my students comment on other students’ presentations, they are very good at finding the problems. And the problems are always exactly what you said, not enough storytelling, no eye contact, not engaging, not straightforward, not humorous. And so it’s easy for us to critique somebody else. But then when you’re doing the presentation, it’s hard. And if you do it for a long time, you get better at it. But your first instinct is not to tell stories, it’s to get the facts and get all the data. So I’ll tell you another story. So I had a student come back, Chad, he was the first Chapman student of mine to work at Goldman Sachs. And he came back to talk to my students. So I coached him before. And I said, “Chad, this is what you want to do. You want to tell a story. And then you want to tell a message.” And it came up and he was shy, even though he was now working at Goldman. And he told some stories, but he didn’t really do it in the way that I thought he should do it. So he told the story, which was funny, he said “at Goldman, everybody uses Excel all the time. But you’re not allowed to use the mouse because the mouse is slower than the keystrokes. And because you’re in this financial factory, it’s important to work as fast as possible,” and his boss would sit behind it and yell at him. “Don’t touch that mouse. Don’t touch it.” Overall I said, “Thank you for coming. I think you could still work, do a better job on your presentation skills.” The end of the year, I’ve given 36 brilliant lectures, [LAUGHTER] and I asked the students in the formal reviews, what was the best part of the class? Half of them said Chad’s presentation… [LAUGHTER] which was entirely story based, and God, I’m just the dumbest person in the world. People love stories.

Jay: I completely agree. And I think that’s what Terry and I bonded over in our very first first teaching experiences. I got asked to apply for some award at UCLA, but part of it was I had to write a teaching philosophy. So I thought, “alright, I’ve been here for a little while, I’m gonna write it down.” And I started listing all this stuff. And later on, I thought, and someone said, “you should try and publish this just in a teaching journal, all these ideas about teaching.” So I do, and I find the references for everything. So everything I do, if it’s about take-home messages, here’s the research that shows this is good. If I do this, here’s the research for this. And I had 10 of these things that were going to help people be better teachers. And the last one, I’m like, “I don’t have any evidence for this, but I just know that it’s good.” And it was “you got to tell stories, you have to reveal stuff about your own personal life, you have to somehow bind the material, the ideas, to your students’ lives.” And just as I was about to submit it, someone says, “I just heard the phrase ‘self disclosure’.” And I just hadn’t looked for that literature, because I’m not an education researcher. And I look and there’s a whole world and thank God all the evidence says that the extent to which you’re telling stories about your own life in ways that are meted out in the proper magnitude and that they are relevant to the stuff you’re talking about. It’s hugely successful. That is what the students remember. And so as long as you can do that, I think, yeah, Chad’s story, “use the keystrokes.” Every time, you can come up with an idea from your own life, if you can make it clear, so the student is going to be able to construct the knowledge later on in their own head, why they were saying that, that’s a huge win for the teacher.

Terry: So I tried this, I try to copy Jay’s stories. [LAUGHTER] So one of my early teaching experiences was Jay was the head teaching fellow for this biology course known as “Sex at Harvard.” And he would give his class, his section on Wednesdays, and I gave them mine on Thursdays. So it was the first time teaching it, and he had done it for many years and was great at it. And so I went to his section on Wednesdays, I took notes, and I would try to replicate. So he would tell these anecdotes, which every time would backfire for me. So I’ll tell the one, the one he is talking about stabilizing selection, why babies can’t be too small or too big, too small, because they’re not viable or too big, because pre-Caesarian you can’t get out of a mother. So he tells the story. “Oh, my brother Patrick, he weighed 13 pounds at birth. But now he weighs 600 pounds. No, he’s perfectly normal size.” So I try that anecdote the next day. Now, it’s saying it’s my brother, Patrick. And a very large person says I was kind of a big baby too. [LAUGHTER] And I’m like, “okay, be careful on the self disclosure.” So, it didn’t go well, anyway, stories, yes, they’re the best. You can’t forget them, right? You’ll never forget that Jay’s brother weighs 13 pounds.

Jay: [LAUGHTER] It is true, and he is normal size now.

Rebecca: We usually wrap up with the question: “What’s next?”

Jay: I’ll say what’s next for me. When I first got into teaching, I thought, “Oh, I get bored of things fast. I’ll do it for a couple of years, and then I’ll move on.” But it turns out that I’m endlessly fascinated by it. Because in order to improve, there is an almost infinite number of things that you can do. So for me, right now, I’m thinking a lot about the pandemic, in many ways, was just this horrible thing if you were a teacher, but we had these constraints that in a few cases, caused people, including me, to discover, “Oh, this is actually useful.” So for instance, the discussion window, and I would have these students who never in a million years would have raised their hands or spoke in class, and they’re typing stuff in the discussion. And other people are responding and they’re engaging. And I thought, “Wow, this is good. That’s what I would have done.” So I’m trying to figure out how to take the various things I learned from teaching in the pandemic. So how can I incorporate a chat window in class so that I get the same effect or things like these just richer learning activities, I had a lot of assignments where they had to go out with a camera and take a picture, just of something and talk about how it was relevant to whatever the lesson was, or these discussion boards were, instead of turning in a paper, they had to post it, and then everyone else got to comment on it. So suddenly, we had this community. So I feel like right now I’m trying not to return to some bad habits from teaching, but instead to say, “Wait, I have ways that I could be better.” So that’s sort of my hope, is that I’m very slow at incorporating new innovations, but I’m a steamroller. So if I could just figure out a few ways to get better now based on those experiences, I’d be happy.

Terry: And for me, Jay and I, our first project together was the book called Mean Genes, which we call a Darwinian self-improvement book. So how to be a better, more effective, happier person by understanding natural selection, biology, etc. And I still think about that all the time. So you probably know this famous 1973 study, to survey people’s self-reported happiness who win lotteries and other people who become permanently physically disabled because of accidents. And you survey their self-reported happiness and after about six months, they get very close back to where they were before the incident. Now, you don’t have the same people, There are methodological issues. But that notion makes sense from an evolutionary point of view, it’s probably the most important chapter in Mean Genes. And it’s to me the most important notion for your own life, which is that the setbacks aren’t permanent. Jay’s a steamroller. And things that seem like they’re terrible, they may very well be terrible, but your ability to recover from them is probably greater than you think at the time. And so for myself, as I said, I’m a terrible parent, I’m always initially disappointed if my kids aren’t brave, and they don’t do the hardest thing first. And I try to do the hardest thing first, every day, because it’s surprising. And the other day, I thought, we had rats in our garage, because we had rat poo in our garage. And so it depressed me, like this is just a horrible thing. So I got up early, I took every single thing out of the garage, I had the exterminator come, and six hours later, I find out there are no rats in the garage, he could tell immediately, it was at a minimum of 12-months old, and my garage was completely reorganized. So literally, from the time that I was the most depressed about my house that I had been in years to the time when I was happiest about my house was about six hours. And it came from not hiding from the rat poo, from addressing it. So what’s next for me is to continue investigation of human nature, how to do things along the lines that we talked about in The Secret Syllabus and in Mean Genes, which is how to make realistic changes in your life, to be a better person for yourself and for the people that you care about.

Rebecca: Well, thank you so much for joining us. I really hope that a lot of first-year seminars and other kinds of opportunities to get this book into students’ hands will happen.

Terry: Well, thank you. Thank you for having this podcast and for having us on.

Jay: Yes, I concur. Thank you very much. It’s been a lot of fun.

John: Thank you. It’s a great book.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

261. Social Justice Assessments

Traditional methods of assessing student learning favor those students that reside in well-resourced school districts while leaving low-income students at a substantial disadvantage. These grading systems also encourage students to focus on their grades rather than on their learning. In this episode, Judith Littlejohn, Meghanne Freivald, and Katelyn Prager join us to discuss a variety of social justice assessment techniques that can help to create a more equitable environment in which all students can be successful.

Judie is the Director of Online Learning at SUNY Genesee Community College, Meghanne is an Instructional Technology Specialist at Alfred University, and Katelyn is an Assistant Professor in the English Department at the Fashion Institute of Technology.  Judie, Meghan, and Katelyn worked together on a SUNY Faculty Advisory Council on Teaching and Technology committee on social justice assessments.

Show Notes

Transcript

John: Traditional methods of assessing student learning favor those students that reside in well-resourced school districts while leaving low-income students at a substantial disadvantage. These grading systems also encourage students to focus on their grades rather than on their learning. In this episode, we explore a variety of social justice assessment techniques that can create a more equitable environment in which all students can be successful.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Judith Littlejohn, Meghanne Freivald, and Katelyn Prager. Judie is the Director of Online Learning at SUNY Genesee Community College, Meghanne is an Instructional Technology Specialist at Alfred University, and Katelyn is an Assistant Professor in the English Department at the Fashion Institute of Technology. Judie, Meghan, and Katelyn worked together on a SUNY Faculty Advisory Council on Teaching and Technology committee on social justice assessments. Welcome Meghanne and Katelyn and welcome back, Judie.

Meghanne: Thank you.

Katelyn: Thank you.

Judie: Thank you.

John: Today’s teas are:

Judie: …I have Lady Grey.

Rebecca: That’s a good one…

Judie: …In my DTL mug.

John: …a nice Desire to Learn mug.

Meghanne: I have iced green.

Rebecca: And Katelyn, how about you?

Katelyn: Mine’s water right now, if it were the evening, I would have one bag of peppermint and one bag of chamomile together, delicious.

Rebecca: Sounds nice and calming.

Rebecca: I have hot cinnamon spice tea.

John: And I have black raspberry green tea.

Rebecca: We’ve invited you here today to discuss your work on social justice assessment. Perhaps, we can start with a discussion on what you mean by social justice assessment.

Judie: So social justice assessment considers factors such as race, culture, language proficiency, socioeconomic status, and ability while working to dismantle systems of power, bias, and oppression in evaluation of student learning. So various approaches including equitable assessment, labor based grading, and ungrading, as they relate to the purpose, process, wording, and structure of student learning assessments are included. So we’re trying to focus on the learning that our diverse students achieve as it relates to specific learning outcomes just to mitigate the influence of dominant norms on our students’ grades. So we’ve all been working together for the last couple of years on a SUNY task group that was part of the Faculty Advisory Council on Teaching and Technology, which I chair. So we’re a subcommittee of an Innovations in Assessment group, and there’s a couple more of us who couldn’t make it today, but we’ve been a really close-knit group, I think, working together for over two years. And we really enjoyed the project, which resulted in a website with all these artifacts on it that people will be able to access. And we’re hoping down the road that we can continue our work, but we’ll get to that later on in this conversation.

John: And we’ll share a link to the overall website as well as your group-specific component of that in the show notes. So this was partly implied in your response defining social justice assessment, but, what are some of the shortcomings of traditional grading systems in terms of equity?

Meghanne: When we were doing our research on this topic, we encountered many drawbacks of the traditional types of assessments that we all experienced all the way up through school and into college, and I’ll share a few of them. One is that the focus is often on the grade rather than the actual learning process and what the student will actually be able to do, and be able to learn as a result of engaging in the education process. They just focus on the grade, “what’s my grade?” and that sort of misses the point. It creates a system where students are compared to each other rather than having the focus be on individual growth and achievement. It also can put students at an advantage or disadvantage based on characteristics such as race, ethnicity, socioeconomic status, disability status, language proficiency, and lots of other characteristics that students themselves don’t have any control over. We found in our research that traditional assessments tend to favor white, affluent, high-achieving students, and that really isn’t who most of our students are anymore. So we really need to remove barriers and create a way for students to accurately represent the learning that has taken place.

Rebecca: So you hinted to this in your response about traditional grading systems comparing students to one another. So thinking about that, what role should students play in determining how their learning is assessed?

Katelyn: I’ll tackle that one, and I want to answer it with a disclaimer to start because social justice assessment is an umbrella term that has all of these different strategies that are wrapped up in it, and each of those approaches, whether it’s ungrading, or labor-based grading, might have a slightly different response to that question. They all share the same goal, that students should not be systematically disadvantaged by the assessment mechanisms, and that we want to increase student agency in the classroom. We want students to be active participants in their own learning, but the actual question of how students might participate in determining their own assessments might look very different depending on who you’re talking to and what approach they use. Maybe it’s literally helping design the assessment mechanisms, the grading contract, grading rubric, maybe it’s creating flexible assignments that allow students to determine what learning is being assessed, or in the case of ungrading, maybe it’s just deprioritizing the assessment entirely in order to emphasize the individual student’s learning journey through the course. So I guess my answer, tentatively to your question, is yes, students should be participants in determining how they’re learning is assessed by the big how, and why is going to differ.

John: As you noted, there’s a wide continuum of alternative grading policies that can fit under this category of social justice assessment. Some of them are not that much different than traditional practices, and others are quite a bit different. One approach, which is much closer to the traditional grading systems that people are already using is a system of mastery learning. Could you talk a little bit about what mastery learning is and how that could be used in the classroom to provide a bit more equity.

Judie: So mastery learning is, instead of assessing a student or evaluating a student with one assessment, and giving them that grade, the students are able to go back and revisit the content and work again on any material that they didn’t understand or try things over again. So it’s an iterative process, and they should get some sort of formative feedback in between attempts so that they can understand what it is they need to work on and focus on. And this way, it’s more equitable, because the students are able to take the amount of time that they need to work on the assessment, they can access any review materials that they need to establish their foundational knowledge and continue on. And it just really helps the students learn and grow. And I think it’s a great way to establish foundational knowledge. I use it myself in all the history courses that I teach, and I just think it’s a great process. If you think about it, any athlete, that’s what they do. So if you’re learning how to play baseball, how many hours are spent in a batting cage, or like on the pitcher’s mound, how many times do you try again, and again, and then again, until you are able to do it correctly, or do things accurately? So I always liken it to use that sports analogy, because I really think that helps people understand that students’ learning… you have to practice and you can’t tell somebody something once and expect them to integrate it into all the knowledge they already have, and be able to recall it instantly. So I just think it’s a great way to level the playing field of students so that when you move on to the next part of your content, they all have the same foundation, and they’re ready to go forward.

John: And by explaining it to the students that way, in terms of a sports metaphor, it’s something that they can pretty easily connect to, and I think it also would help to promote a growth mindset, which we know is effective in increasing learning as well.

Rebecca: Another assessment strategy one might use is minimal or light grading that falls under this social justice umbrella, and is a bit different than mastery learning. Can you describe what minimal or light grading is?

Meghanne: Yeah, I’ve seen this described in a couple of different ways. This isn’t something that we really included in a lot of our research, so I kind of looked this up just a little while ago and it’s very interesting. And one approach is more on like the whole course level. And there’s another approach that can be taken on an assignment level. So for an entire course, what an instructor may do is that they would assign assignments throughout the semester, but most of them would not be graded, they would be used as like a conversation piece. And they would be discussed and gone over during class, which would then provide opportunities for the students to seek clarification and for the instructor to provide feedback in the moment. So then the assessment then becomes part of the learning process. So then when there are a small number of assessments that are given for a grade, then when the students get to those assessments, they’re not as intimidating. They’re things that they’ve done with their classmates, they’ve done them with their instructors, they’ve done them in class. So I think it’s a very interesting strategy because it removes a lot of the anxiety that students may have around assessment, because it’s just something that they’ve done in their class. Another take on this that I’ve seen is, on an assignment level, something like a paper, something that may require a lot of revision, where when the professor is grading that assessment, they would maybe not take the time to go through and mark all of the grammar and spelling and mechanical errors, but maybe they would look at a section of that, maybe point out some things the students are doing over and over again, but not mark up the entire paper, but just say, “Okay, these are the things you need to pay attention to that are recurring through your paper.” And then as they read and grade that student’s paper, they focus more on the message that the student is trying to convey and the ideas that they’re sharing, rather than the mechanics and the grammar and the spelling.

John: And one common thing I think, to both mastery learning and minimal light grading is that the goal is to provide students with feedback. In some cases that can be automated. Mastery learning systems involve some degree of automation, sometimes by textbook providers, or perhaps adaptive learning systems, or it could be questions that you put together. But if you’re going to provide feedback on writing, it can require a lot more time. And a minimal light grading approach allows faculty to provide feedback on the most important things without taking up as much time to allow faculty to provide feedback on a wider range of topics, which, again, is I think, to some extent in the same sort of spirit.

Rebecca: Light grading can help not intimidate a student with too much feedback. If you see just a paper completely marked up, it might feel like there’s no possibility for moving forward or revising. But emphasizing what’s most important to change, or most important to focus on can help a student prioritize. And this can be really important to someone new to a discipline who might not know what’s most important.

Katelyn: I’m so glad you said that.

Meghanne: There’s an element of trust there as well, because if we point out what a student needs to focus on mechanically or grammar wise in a small part of that paper, then they can be trusted to then use their judgment to go through it and read it more carefully, and then make those edits based on the feedback that they had received. So it is visually much less intimidating. Plus, it might be a motivating factor for some students too that their professor is trusting them to be in charge of that revision.

John: Another type of social justice assessment involves contract grading. Could one of you talk a little bit about how contract grading fits into this category of social justice assessment?

Katelyn: Sure, I think contract grading is one of those terms that’s gaining some broader popularity and recognition. So it’s probably a term that may be pretty familiar to a lot of instructors at this point. So maybe it doesn’t need a lot of explanation. I’ll just say there’s a couple of different models of contract grading. In some cases, the instructor might provide that contract at the start of the term. In other cases, the instructor and students would be able to negotiate that contract collaboratively together at the start of the term so that students have more of that active stake in the contract itself. Generally, the grading contract would lay out certain requirements which students would need to fulfill to receive their desired grade. And that might include requirements related to attending class or conferences, completing low-stakes assignments, completing major assignments, maybe some page- or process-based requirements. But the bottom line is that the contract gives students a clear picture from day one of the work required by the class so students can look at that contract and know exactly how much work they’re going to need to complete from day one, to get the grade that they really want to receive in the course. I think the additional benefit of contract grading for our conversation is that it decouples grades from assessment so students have more space to take risks in their work rather than aiming for correctness. And on the faculty side, faculty can respond to the content and spirit of the students work as opposed to justifying a grade. I think most important, though, because this system privileges students who are investing the time and effort into their learning, all students have the same potential to earn a high grade in the course regardless of their knowledge or ability with the subject matter prior to the start of the course. So to use another sports metaphor, it works to level the playing field on day one for students who may have very different levels of preparedness and experience with the subject matter.

Rebecca: Another strategy that folks might use, which we’ve certainly talked about quite a bit on this podcast at various times is peer assessment. Can you talk a little bit about what that looks like and how that fits into this social justice model?

Judie: So peer assessment, or I tend to call it peer review, helps to build student investment in writing, and helps the students understand the relationship between their writing and their coursework by helping them engage with the writing in a way that encourages more self reflection and works to help them build their critical thinking skills about their own work. And I think it also helps the students learn from one another, because they’re sort of trying to evaluate their peers’ work against the requirements for the course. But then you also look at your own writing in a new perspective, and you learn from what you’re seeing your peers write and from the feedback that you’re receiving from your peers.

John: Might students perhaps take feedback from their fellow classmates a bit more seriously than they do feedback from their instructors.

Judie: A lot of students self-report that they learn more from this peer review activity, because they’re trying to identify and articulate weaknesses that they’re seeing in their peers’ papers, and also in their own. And I think trying to incorporate feedback from both their peers and their instructor into their own work, I think, just helps raise that awareness and any kind of feedback that’s constructive, as they think about it and reiterate it and rewrite their work. It just helps with their critical thinking. And I think just raise awareness of how they write, and maybe they can be more thoughtful about what they’re writing going forward. I think they also, if they question their peers, say “How did you come up with this?I love this idea,” then they can apply some of this, that they’re learning from their peers to their own work, too. So perhaps that’s what you were getting at John, when you asked that question was, they may benefit more from their classmates telling them how they came up with their ideas than from their instructor just dictating what the expectations are.

Rebecca: I would expand the model to include not just writing but also other creative projects and things. It’s certainly a practice that’s pretty common in the arts, for example, to do peer review of student work.

John: And they also get to see what their peers are doing, which can serve as a positive role model. When students see that other people are doing something that they hadn’t considered doing, it could serve as a way of improving their work.

Katelyn: I think a lot of students come into the classroom thinking of their teacher as the sole reader or audience for their creations throughout the course of this semester. So anytime we can expand those audiences and have students thinking rhetorically about who else might be the consumer of their work. I think that that can benefit our students in really important ways.

Rebecca: It also seems like it’s a good opportunity to formulate community around an activity like that.

Katelyn: Absolutely.

John: One of the other areas you address with this group was the topic of labor-based grading, could you talk a little bit about that?

Meghanne: Yeah, labor-based grading removes the focus from the end product assignment and shifts it to the process of creating that piece of work. So students are provided with feedback throughout the process regarding their labor or the work that they put in. And they’re given opportunities to continue working to improve what they’re producing, and to achieve a desired grade based on a contract sometimes, so there is some overlap with contract grading, but not always. There typically aren’t penalties for students who revise and update their work, because that’s part of the learning process. And it really helps students determine what their end grade may be and how much effort they want to put in, because often, they will be given some sort of guideline for what different grades may be achieved based on certain levels of effort, or certain levels of work that are completed. And also there may be opportunities to grade based on completion rather than more of a subjective sort of qualitative grade.

John: So do you mean like using a light grading or minimal grading where you either completed satisfactorily or you haven’t, and as long as you complete a certain number of assignments or activities, you achieve that grade,

Meghanne: That or also if there’s criteria, like a rubric, and they hit all of the criteria, then they receive full credit.

John: Which becomes, actually, I think, a form of specifications grading.

Rebecca: And then one other model that you’ve talked a little bit about already today is ungrading. Can you expand upon that a little bit more?

Katelyn: Yeah, so ungrading works to deprioritize numerical grades or even attempt to eliminate them entirely. So I hope I’m not speaking out of turn when I say, I think that this is the most controversial of the approaches that we have been researching, it tends to get the most pushback from faculty because it is so different from what we have often been taught or trained to do. So instead of focusing on those numerical grades, instructors are encouraged to focus on providing learner feedback that encourages growth. Okay, I have a quote that is from an ungrading expert I’d like to share. This from Sean Michael Morris and he says, quote, “at the foundation of ungrading, lies something that could change school entirely. A suggestion that ranking and evaluation and the concomitant expertise of the ranker or evaluator is entirely an optional way of viewing things.” And I’m going to end the quote there because I think that that important kernel is that ungrading works to dismantle the hierarchy of the classroom and refocus the attention on individual student learning is an approach that requires a lot of trust between student and instructor, and a lot of student buy-in as well. Students have to be invested in the learning that’s going to happen throughout the course itself. And in a completely ungraded classroom, student grades might be based simply on a final student reflection, or even a one-on-one conversation between teacher and student about the grade that the student has earned. But because ungrading really rejects transactional grading systems, the final grade is more of an afterthought than an important outcome of the course, much less important than learning that’s occurred throughout the semester.

Rebecca: So today, we’re recording on August 9, James Lang posted on Twitter about how deep the system of creating actually is that there’s even things like discounts for insurance, for good students, or good grades. And that it’s really challenging to overcome a system that’s so ingrained beyond just our education system, but into many other systems as well. So I think that that, in part, is why there’s such a strong pushback on this particular method.

John: And we’ve always done it that way, at least for the last century or so.

Rebecca: Change is hard.

Katelyn: Yeah, I think that the traditional grading system is really embedded into not only academia but outside of academia as well. And even within a class that takes an ungrading approach, we still face that question at the end of the semester of “Well, what’s the grade going to be in the system?” because we don’t really have the option, at least at most institutions, to say, “No grade, job well done.” At least at my institution, I still have to put in a letter grade for the student. So we can work to reject that system as much as we can. But at the end of the day, we’re still operating within that same structure. And maybe that’s a question of what’s next, right? Like, are we going to see one day a future where more universities embrace this idea of learning for the sake of learning as opposed to learning for the grade? I don’t know.

John: One of the other things you address on the website is how perhaps the use of authentic assessment or UDL types of assessments might improve equity by providing a more equal playing field for students. Could you talk a little bit about how going beyond the traditional term papers and tests might provide a more equitable way of assessing students’ learning.

Judie: I think anytime you use authentic assessment that helps, or generally it allows the students the opportunity to demonstrate their learning in the way that works best for them. The students are writing a term paper, for example, they can write the paper the traditional way, or they can give a presentation or record a presentation, and still provide their citations and so forth at the end. Or they can do something visual, some sort of a PowerPoint or a nice visual display of the topic and again, cite their sources and explain their images to the group so that people understand how they’re meeting the learning outcome. And I feel like that’s just a good way if people are struggling with language, if people are just struggling with writing in general, I think that this levels the playing field, because it gives everybody an opportunity to really show their knowledge and shine and not just pigeonhole themselves into one more paper or one more multiple choice test, if they have test anxiety. Some of our traditional forms of testing or final assessment just set students up to fail. And allowing students to choose to demonstrate their learning in a way that they’re good at sets them up to succeed. And I think that’s what we really want at the end of the day. And of course UDL principles, those are Universal Design for Learning, and that does include equity in its heart. So that would definitely help to keep things equitable in the classroom. If you’re following UDL.

Rebecca: The multiple forms you were just talking about is a great example. [LAUGHTER]

Judie: Last semester, I had a student who, they’re supposed to do a blog post, and the student instead of writing a blog post, he made a video and he did it three different times. So one is on World War One, one’s on World War Two, and the third one was on revolutions, and so, this student stood in front of a whiteboard, and he had his camera set up so he could film himself. And he had his iPad in his hand. So he talked about a battle, say, for example, and he would draw it out on the board. And then he would show his citation on his iPad. And then he had other citations typed up and taped to the whiteboard. And he went on for 15 minutes, and just was making sure he explained things again, and drew little examples. And he was so animated, and so excited about his topic. And you’re not going to capture that on a written exam, or even in somebody’s written paper. It was just tremendous the way he was able to show all that he had learned and all that he was interested in, and the extra research that he had done, because he felt the freedom to pursue this topic, because he knew he was able to express it the way that suited him the past. And it was just amazing. So I think anytime we can incorporate these things, and I understand that there are times when, according to your creditor, or people have to sometimes sit for a specific certification, it doesn’t always fit, but I think if you can fit this type of assessment in, it is definitely worth it. Because just to see the joy in students when they can explore and expand their knowledge, and then feel confident in demonstrating that to you, it’s just tremendous.

Rebecca: I love the flexibility in demonstrating knowledge and understanding and skill sets because in some of our traditional methods, we are arbitrarily assessing something else. So we may be arbitrarily testing how well you can take a multiple choice test or how well you can take a test within a certain timeframe, or how well you can write, whether or not that’s actually the topic. So if I’m learning about history, there’s some learning objectives I’m trying to meet related to history that may or may not include writing. And if writing is not one of those outcomes that we’re hoping for, then we don’t need to be assessing it.

Judie: Exactly. He did this thing on medical advancements in World War One, it was just tremendous and he was so charming, because he just was so wrapped up in it that you just had to root for the guy. It was good.

Rebecca: I love that. So for those of us who may want to move towards equitable grading systems, what are some initial steps we might take? Because it could feel really daunting if you haven’t ventured down this path before.

Meghanne: Yeah, if you are not interested in overhauling your entire grading system, just to try this out, a nd to make your assessments more socially just, there are some adjustments that can be made to existing assignments. And really, the important thing is to consider the learning objectives and really think about what needs to be graded. So one of the things that we’ve talked about a lot in all of our different presentations that we’ve done is whether or not to grade for things like grammar and spelling, and mechanics, and English language proficiency. So in an example, like a discussion board, when you’re really interested in what the students have to say, and their interaction with each other, and the questions that they asked, does it really matter if their grammar and spelling is perfect in that instance, if they’re having a great conversation on a topic, and they’re learning from each other. So that’s one thing that we could suggest. Another is thinking about just the fact that sometimes students have challenges in their lives. They’re human beings, they have families, they have jobs, many of our students are athletes, and they have to travel and they have games and something like flexible due dates is very, very helpful for students because then they’re able to complete their work, certainly within a reasonable timeframe. But if those dates are a little bit more flexible, and they have access to those assignments in the learning management system beyond the actual due date, for instance, then that gives them the ability to complete that work without being penalized. So another mechanism would be in the learning management system, when students are taking quizzes, would be allowing backtracking, allowing students to go back and check their answers, that sometimes is a setting that a lot of professors really rely on, to try to avoid cheating. And as an LMS administrator, that is something that I see a lot. And I think that that can really be harmful to students, because many of our students are told to always go back and check your work. And if they’re not allowed to go back and check their work, that can be very frustrating. And also forcing completion is something that I would recommend turning off because again, that can create test anxiety. And often I think when completion is forced, there’s also a timer. So I think if any timers can be removed as well, then that does a couple of things. It can help remove testing anxiety. But then also, if there are students who require extra time due to a disability accommodation, then the professor at that point doesn’t have to go in and adjust all of the LMS settings for those students, because it’s already open ended and everyone can have as much time as they need to complete that assessment. So it really is just important to look at what the learning objectives are and what actually needs to be assessed. And the goal is always to remove barriers. So another thing that can be done is to just ask students, have a conversation about it, and find out what barriers they’ve experienced.

John: At the start of this. You mentioned the website that you were creating, could you talk a little bit more about what resources are there and how that might evolve over time?

Katelyn: Yeah, so the website, we have been slowly adding resources to over the past two years. And at this point, it’s becoming a pretty robust little outlet for people interested in social justice assessment. So, you go to the website, you can find an overview of the big picture theory of social justice assessment, as well as the various approaches that we’ve discussed today. We also have a really pretty large bibliography of resources for further reading for people who want to learn more about any one of these topics. And we’ve been working to develop a collection of sample assignments from faculty across SUNY. So we’re still working to collect additional sample assignments from faculty who might already be implementing some of these strategies within their classrooms. I think the more we can share those assignments with one another, the better off we’ll all be. I think a lot of us are doing social justice assessment in small ways in our classroom without realizing it. So the more we can share those resources and that knowledge, the more hopefully we can get people on board. So, hopefully, we’ll be able to share that link in the show notes. And people will be able to check that out.

Rebecca: So we always wrap up by asking: “What’s next?”

Judie: So for our little group, one thing that I think might be next for us is SUNY is updating the SUNY general education requirements that are mandated with the completion of any SUNY degree. And they’ve added a requirement for equity, inclusion, and diversity. So I’m hoping that our group can help contribute resources to that effort, and our website could be one more place where people go to for information on social justice assessment so that they can incorporate those into their courses that are designed to meet the DEI requirement.

Katelyn: Well, I’m gonna go take my one-year old to the pool. [LAUGHTER]

Judie: Nice.

Katelyn: I think, big picture, though, the “what’s next” I want to just give is, I hope that we’ll start to see more institutional support for some of these approaches. I think that there are still a lot of barriers, particularly for contingent faculty who want to embrace some of these practices. So I hope what’s next will be more departmental institutional support for this: more time, more resources, etc. But yeah, my personal what’s next is I’m gonna go enjoy this beautiful day.

Rebecca: Meghanne, do you want to add anything?

Meghanne: Sure yeah, at my institution, I am sharing this information, pretty much any chance I get, I’m meeting with our new incoming faculty in a couple of weeks. And this will be one of the topics that we discuss. And I’m also co-chair of our universal design for learning task force. And we have a few events and projects that we’re working on to spread the word on UDL, and also innovative assessments and social justice assessments as well.

Rebecca: Lots of great things coming and some really wonderful resources that you’ve shared today. Thank you so much for joining us.

Katelyn: Thank you.

Judie: Thank you for having us.

Meghanne: Yeah, thank you.

John: And thank you for all the great work you’ve done on this over the last couple of years and the resources you’re sharing.

Judie: I would just like to say that Shena Salvato is also in our group. She’s at Cortland, I believe. And Chris Price from SUNY is in our group, and they are missed today. They’ve been with us for all our other presentations. I know that Shana in particular wants to get the band back together and have some more meetings going forward so we can keep working together. And it was really good to see you guys again.

Katelyn: Likewise.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

257. PsycLearn

Adaptive learning platforms provide each student with a customized learning path based on the student’s individual learning needs. In this episode, Anna Yocom, Linda Goldberg, and Alan Strathman join us to discuss how the American Psychological Association has developed adaptive learning packages for core psychology courses.

Show Notes

Transcript

John: Adaptive learning platforms provide each student with a customized learning path based on the student’s individual learning needs. In this episode, we examine how one professional association has developed adaptive learning packages for core courses in their discipline.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are all involved with a psych learning project, which we’ll be talking about throughout the podcast. Let’s have each guest introduce themselves.

Anna: Hi, I am Anna Yocom. I am a senior lecturer at The Ohio State University in psychology and I am also a Content Manager for PsycLearn.

Linda: And I’m Linda Goldberg. And my role with the PsycLearn project is that of product evangelist.

Alan: Hi, this is Alan Strathman. I’m a content manager for PsycLearn and was a professor of psychology for a long time at the University of Missouri.

John: Welcome, everyone. We’re really happy to talk to you about PsycLearn. I first heard about this when I was working on a project at SUNY dealing with adaptive learning solutions. And we had seen some write ups of PsycLearn so we want to find out a little bit more.

Rebecca: Today’s teas are:… Alan, are you drinking tea?

Alan: I’m drinking coffee.

Rebecca: Oh, a coffee drinker.

Alan: Sorry.

Rebecca: I see how it is. [LAUGHTER]

Alan: I had tea this morning this morning.

Rebecca: Oh, that’s good. That’s good.

Rebecca:: How about you, Linda?

Linda: I’m having an iced tea this afternoon. It’s hot and muggy outside, so something citrusy and decaf.

Rebecca: …and perfectly refreshing. [LAUGHTER]

Linda: That’s right. And sweetened.

Rebecca: Awesome. Anna?

Anna: I had tea earlier, but I have switched to water.

John: And I’m drinking spring cherry green tea.

Rebecca: Nice. I have some Scottish breakfast. Still in my pot today.

John: We’ve invited all of you here today to discuss the PsycLearn adaptive learning platform developed by the American Psychological Association. Could you tell us a little bit about the origin of this project?

Linda: Okay, I’m going to start with that. And there are a couple of overlapping facets to the origin story, so to speak. One is really a desire to expand APA’s publishing program. APA has a very rich publishing program, which includes publishing of over 90 academic journals, a very robust books publishing program, where many of those books are aimed at clinical practice, as well as books that might support the professional researcher, books that might support self help, and so on. And we found that there was an opening for us to make a contribution to undergraduate academic instruction and contribute to that portion of the path toward the profession. We also were recognizing as we contemplated this new project that there was a tremendous amount of disruption in the higher educational publishing landscape, a lot of cost pushback, a lot of desire among publishers to leverage technology. And we thought that perhaps not simply for delivery of content, but to improve the learning experience and to improve efficacy that we saw there might be some opportunity there.

Alan: Yeah, also, let me jump in. I think the developers also saw this as an opportunity to work toward some of the goals that APA has for the field of psychology, including, for example, utilizing psychology to make an impact on social issues.

Linda: And the field of cognitive psychology is one that tells us a lot about how we learn. So we felt like we could draw from our very own field in development of our final product,

John: Which courses have been developed for this platform.

Linda: So we started with a course for research methods and psychology and added a PsycLearn statistics for the behavioral sciences to that. These two courses are typically required for all majors in the discipline. And they also represent the most challenging content for many students. So we thought that seemed like a good place to start.

Rebecca: Can you talk a little bit about how this material was developed?

Alan: Sure, we use a backwards design process where we start by identifying what we want students to learn in each module and a module is similar to what would be a chapter in a regular textbook. These outcomes that we have in mind become Learning Objectives, and they really guide the content development. Learning Objectives are developed through collaboration, a conversation, maybe a hashing out between the subject matter experts and the content managers, and are influenced by a variety of factors, including how well the learning objectives address APAs guidelines for undergraduate education. And then there is a rigorous review process where other experts in the field provide feedback and often suggest additional learning objectives

John: Was there much of a challenge getting agreement on learning objectives. I know in many disciplines, there’s a lot of discussion of what should be in the core material in the discipline? Has psychology been able to overcome that a bit by agreeing on these standards?

Alan: Well, I think it is a challenge in every case, because everybody has a sense of what they think should be taught and what is essential for students to learn. Like I said, I think it’s hashing out. It’s sometimes a little bit of trial and error, where we identify a learning objective, and then start writing about it. And then at some point, we may decide that’s not the right objective to have. But I think in the end, the subject matter experts and the content managers do a really good job of coming to some consensus on what we want to teach.

John: Was the content in the courses, the actual readings and other materials, developed by the people who created the package?

Alan: Absolutely. We are a digital first product, with original content developed by experts in various sub disciplines in psychology. And this really gives us a big advantage I think over print textbooks. So starting from scratch, in an entirely digital environment, we can develop a seamless presentation, where we start with learning objectives, we present content, we design activities and interactive exercises, and then have assessments. And we can be sure that all those things are aligned with learning objectives. And we don’t have to try and retrofit learning objectives into existing content. And then just hope that assessments that we have address the new learning objectives. So I think being able to work in a digital environment allows us to have this really seamless presentation where we start with learning objectives and we go all the way through assessment, and all those things match.

Rebecca: You’ve hinted a little bit about maintaining the content, given that it’s in a digital platform. But can you talk a little bit more about how you maintain that or what that review processes is or how that gets updated on a regular basis?

Alan: Well, that’s a good question. One issue for us is that we’re still so new, that we haven’t had semesters and semesters of feedback from students that we can use to make revisions. But what is really exciting is that whenever we decide that we want to make a change, we can do it in real time. Think about a print textbook, whenever they decide that they want something to be different, they might wait three to five years for the opportunity to make those changes. With us, since it’s digital first, we decide something needs to be changed or updated or revised and we can make those changes in a really timely fashion. And I think it really helps to keep our content current.

John: And if you find that students are not achieving the learning objectives, you have the ability, at least, to make some changes, either redefining the objectives if perhaps the objectives didn’t work too well. Or if the learning materials weren’t quite aligning with that. How often is that type of change done?

Alan: Well, so far, we’re still into the development process, that we’re making those changes just all the time. As the products get older and more mature, I think we’ll need to do that less. But again, we’ll still have the opportunity to do it as often as we’d like to.

Rebecca: John mentioned at the beginning of the episode about his interest in this project related to adaptive learning. Can you talk a little bit about the adaptive learning component of this project and the platform that you use.

Anna: The platform that we use is CogBooks, it is a platform, I think, is really unique compared to other textbooks, because it allows students to engage in their learning, and really direct their own learning pathway. And that’s based on their understanding of the content. So instead of going through a whole chapter, and then maybe quizzing yourself or testing yourself, after each content page, they get some formative questions and they get to make their own assessment. They get to say, “Okay, did I understand that? Was I not too sure? How did I do on those questions?” And then they can choose whether they want to get that support material or not, which we think of as like, “Would you follow up and ask your instructor a question.” That’s how we think of this support material. So if a student is very confident with the material, they’re essentially going to get a different path than a student who might need help in a couple of different areas or a student who might need help in a lot of areas. There’s just more examples or more practice. So it can look a little bit different for each student, which is pretty cool.

John: Linda mentioned earlier that psychologists have done a lot of research and cognitive scientists have done a lot of research on how people learn, how have some of those principles of how we learn been built into this platform?

Alan: Well, Linda is certainly right. There is a lot of research on learning science. And because we are psychologists, we’re familiar with the research, and use that knowledge to help students learn throughout the product. I think the centerpiece of our efforts to use learning science is the inclusion of metacognition, that is helping students evaluate their own learning, helping them identify what they know and what they don’t know. And when they identify what they don’t know, then they could spend a little more time learning that particular topic. Often students get to the exam, and they haven’t really done a good job of testing themselves, identifying what they know, and what they don’t know. And the first time they realize they don’t know something is when they get to the exam. And that’s obviously not the right time to get there. And so we’re able to identify ways that we can help them evaluate their own learning. We help them in this process by incorporating those learning strategies that we know are supported by research. We present concrete examples throughout. We have frequent activities and assessments. Students get practice retrieving content from memory. And these opportunities are spaced just as the research suggests they should be. They have interactive exercises. They get practice in the elaboration, or explaining content to themselves in their own words. And really the process of taking material and from the jargon in which it’s presented to explaining things in their own words, is a really useful process for students to engage in. And we could have a whole podcast just on principles of learning science that PsycLearn incorporates.

Rebecca: Can you talk a little bit about what the student experience is like for students that are using the platform? So Anna, you hinted a little bit at this with some of the adaptive learning components. But what does it look like from: I entered this space from the beginning, and now I’m interacting with this over time.

Anna: So students really, I think, approach PsycLearn differently than a printed textbook, again, not only just the fact that they would go through a printed textbook, and then maybe ask themselves some questions, probably skip over a lot of reading, because maybe it’s not very engaging or easy to get distracted with print textbooks as well. But one of the really interesting things is that they’re engaged with the content pages within them. So they’re not just engaged outside of content, like “Oh, refer to this video,” or “you can reference this activity” or “try some completely different platform to help with your reading….” but within the content pages, they get activities in which they’re asked to reflect on the course material. How does this connect to things you already know? So they get to respond to those things within the content pages. They get practice formative questions, formative meaning that it’s not just did I get this right? Yes or no. But they get feedback. So they would find out what a suggested answer was to a short answer question. They can compare that to their own. With a multiple choice question, they get follow up on why that is the correct answer, not just “Well, if I’ve missed this a couple of times, and they get multiple chances to respond, but they get some explanation about why that was the correct answer.” And so we have video questions, we have matching exercises, we have short answer, we have multiple choice questions, different drag and drop. So we’ve tried to just vary the kinds of interactions that students would have, the ways that they can practice answering questions. And this variety really helps maintain interest throughout. So they get to engage it in lots of different ways. It’s not just a rote memorization, but they’re actually applying the material as they go through each module.

John: So you mentioned formative questions. Are there are some assessment tools built in to evaluate students’ learning for grading purposes, for example?

Anna: There are. Yeah, so there are summative questions. And we try to have them associated with those learning objectives that Alan had talked about earlier. And so after maybe one, two, or a small group of those learning objectives are covered, there is a short sample, maybe 5…10 multiple choice questions where students have a few attempts, and we want them to get to about an 80% accuracy. And one of the nice things is if they don’t on their first attempt, they can actually go access those support pages then. So maybe they skipped over them earlier. And they thought, “Okay, I got this, it’s fine. I understood this page.” And then they get to these questions later on, these mastering the content and they’re like, “Well, wait a minute, I was missing some of this.” So they get another chance to go back to those support questions, and try it again. So then hopefully, they can get those right after three attempts. We don’t want them to be punitive. We want them to get engaged with the material and get to that mastery.

Rebecca: So now that you’ve talked a little bit about the student experience, can you talk a little bit about how this supports instructors? What kind of feedback is provided to instructors and maybe how instructors are using this in their teaching.

Anna: One of the things I really like, we all know, anytime that we can even maybe think back to our own experiences in classrooms, it’s a struggle getting students to read. But a lot of these classes, I mean, really all of our classes, if we’re assigning some sort of book, some reading material, we want them to read it. So how do we get them to engage? Well, that’s where the accountability comes in. So we’re giving them this tool, and then their instructor can see, “Okay, they’ve made it through this material, they engage with this.” So the instructor could choose to then reiterate that material in class, or they could choose to maybe take an active learning approach in class. And okay, now that you’ve had this background, everybody made it through, we’re going to apply it. And maybe not everybody got to 100% on every question, but maybe we got to about over 90% completion, which is what I would typically see in my classes. So you would have students who, even after a few attempts, would sometimes miss some of those mastering questions, but generally, it was pretty high. And so I could see that before I went into class. Okay, so students, as a whole, on average have got this. So I can take it a step further, where I can go back to those questions that were really challenging. So you get to track progress through each module, you can also dive a little deeper with individual students. So a student comes to you and says, “I’m really struggling,” the instructor can go in there, they can look at their pathway. Were they accessing the support material? Did they take those quizzes a few times? Where were they spending their time? Or if the instructor wanted them to cover it on let’s say, Tuesday night, were they doing it all Tuesday at 9pm? Right? Like you could actually see all of that. So you can kind of work with them like, “Well, I see you started this that night, let’s think about how we could maybe back that up. Like maybe we should pace this out at least a few days ahead of time, which is a really nice way to work with students.” And it also syncs with the LMS, the learning management system, so it really takes that burden off of instructors as well. So it’s talking to the platform that we get there. So we get this easy back and forth, which is really nice. So we can nudge students along with that, I would often encourage students to visit those support pages if they were struggling, like, “Look, I know this is really challenging,” or, “Hey, there’s more practice, if you just want a little bit more practice, you can go to some of those support pages to get that.” And the other nice thing is students can actually email right from there, which I think is pretty unique. So if a student emails me, I could see okay, they were having problems in statistics with variability and standard deviation. And it will take me as the instructor right to that page. And so then I can see that and I can say, “Oh, well, why don’t you try these examples here,” which was really nice. Instead of just a student saying, “I don’t know, I don’t get this,” they can let you know exactly where they’re struggling. So it provides a lot of options for instructors.

John: And do instructors get aggregate statistics on what areas students are struggling in addition to data on individual students?

Anna: Absolutely. So you can see that on the dashboard. You can see if there were chunks of mastering questions, for example, or a specific question that students struggled with. So you could choose to review that and maybe even go over some of those in class or go over specific topics in more detail.

John: How are instructors combining this with other face-to-face learning activities, for example, in face-to-face classes. As I understand it, most people use this as a textbook replacement, except a much more powerful replacement to a textbook. What other things will people often do to provide more of a sense of community or a social component to the learning?

Anna: Yeah, so, I think a lot of instructors are using it, just as you said, as a textbook replacement. And that’s how I have used it for statistics in particular. So it was taking the place of my other textbook, which actually was a print textbook that was online. And so I don’t feel like there were any losses. But we were gaining the ability to say, “Okay, we’re all on the same page. I know that everybody’s at least had some exposure to this. So now let’s talk about it. I can see where you were struggling. So I know we’re going to need more examples here. I know we’re going to need more follow up here.” Or “why was this question challenging? Why was this hard?” So I think drawing on those parts are how we can maintain those social ties and the social interactions we want whether it’s online, whether it’s face to face, or in any capacity.

Linda: I’ll just go ahead and point out as well that part of the content package that we’re making available to instructors who adopt, PsycLearn course material is a set of student activities that sort of exist outside of the platform itself and are available for instructors to assign or to bring to the class. They’re typically designed again to align with specific learning objectives and to give the instructor options for an in- class activity or an additional assignment activity. And that’s accessible to the instructor when they’re using the dashboard.

Rebecca: Sounds like some really great support materials. Can you talk a little bit about how this platform has affected equity gaps that we might be seeing?

Alan: Sure, I think there are two ways that PsycLearn is addressing issues of equity. First, we are designing PsycLearn to be culturally inclusive, that is we work to make sure that through examples, images, photos, figures, every student can see themselves in the product. Plus, our goal is to include the research of diverse psychologists and psychologists of color. And then I think the second way that we strive to be equitable is to design for universal access. We want the product to be accessible to all students, and so we make great efforts to do so. We use typography that’s particularly accessible. We incorporate transcripts and closed captioning, we have text-based alternatives for non text elements. And our efforts are well beyond what the minimum guidelines would be, well beyond what typical expectations would be, and I want to say that this is because we value this process so highly. We are keenly interested in making an accessible, inclusive product, and I think we’re really working hard to do that.

John: Have there been any studies of the overall efficacy of this program compared to alternatives?

Linda: Well, we have conducted some impact surveys, surveys of student users to gauge their impressions of the impact of PsycLearn on certain important aspects of their experience. Through surveying students, we looked at the impact on their knowledge gains, on their confidence in their ability to use that knowledge, and on motivation to complete assigned activities. And on the questions that we asked, to sort of tease out each of these thematic areas, we did receive very positive response. Now, these are, albeit, impressions of students. That’s sort of the first line of study available to us. And we feel pretty motivated ourselves by what we’re learning, by what we’re hearing from students. These surveys did also include some open text entry questions. And very interestingly, some of the criticisms that we heard were often this is a lot of time, this is a lot of work. But by the same token, we’ll hear in the very same response, but it’s very worth my time, and it really helps. And I feel confident when I go into the class, and I already know what my instructor is going to talk about, I feel more relaxed, I feel more able to pay attention. That kind of response has been very rewarding to receive from these student surveys.

John: With some other platforms, there have been studies that have looked at the impact on equity gaps, and they relate very much to what you just said, that basically, students come into our classes with very different backgrounds. Some come in with a very rich and strong background from prior courses, others come in with a somewhat weaker background. But to make it through an adaptive learning platform, the students who come in with less background can acquire mastery, but it takes a bit more time and effort. And the students who come in with a stronger background can race through a little bit more quickly. But much of the research does suggest that those equity gaps tend to be reduced. But it is at the cost of additional effort by those students who come in with a little bit less prior knowledge or prior experience with some of the material. And so I think that’s one of those real strengths of adaptive learning platforms compared to other formats like textbooks and lectures and so forth, where everyone is expected to move at the same pace. But as Chuck Dziuban, on a previous podcast has said, in a traditional course, everyone spends the same amount of time with the material, but they learn different amounts, when they’re working with adaptive learning platforms, they have to spend different amounts of time, but they all can achieve the same learning outcomes, they can all reach the same outcomes. But it may take more time to do so.

Linda: That’s certainly our hope, and some of what motivates the work that we’re doing. I feel like having respect for the time that our students need to spend is top of mind. We need to make sure that what we’re asking them to do is a good use of their time. And therefore this mode of delivery and the kind of content that we make available through the multiple activities and so on that Anna was describing. We’re giving our students opportunities to do that self testing that reiterative process and not simply trying to digest a narrative.

Anna: I always think back to a student I actually had in class a number of years ago, and this student went through the same class multiple times. And it was a class that was needed to get to other upper-level classes and was very frustrated, of course. And then we came to find out that this student in the multiple times of taking the class, the book was listed as required, but with either a digital or print textbook, you really don’t have any way to monitor that. You don’t know if the student’s using the book even if they have it. But this student had never got the book, just never occurred to them that, okay, this could be a really valuable tool. And so that’s really a benefit. I see. Like you were talking about equity, let’s give them this tool. But let’s make it something that’s valuable, that can really help them and if they need that added support, they’ll get it. But kind of level the playing field a little bit.

Rebecca: How do you see this platform evolving over time.

Linda: So we are able to work fairly closely with our platform partner, CogBooks. We’re able to contribute to their roadmap planning to some degree. We’ve collected feedback through a variety of channels, also, not only the student surveys that I’ve mentioned, but we also, in the past year, conducted a qualitative study that included student interviews and observations of students who were assigned to use PsycLearn. And this exercise gave us really valuable insight into how and where students focus their time and attention. And so for instance, through that process, we were able to get some real good insight into how they approach the summary of each module. And as a result, we’ll be able to give some greater attention, we’ve got some plans to enrich and make more accessible, the content that we placed there, because we were able to learn that they highly value that for later test preparation. So that’s just an example. We’re hoping to be able to continue to do those observational and interview studies.

Rebecca: So I’m really curious for each of you to answer this question. But what makes you really excited about PsycLearn?

Anna: I’m happy to go first, because I am a very energetic instructor. I will be at the front of the room, and I bring a lot of energy. And what makes me excited about PsycLearn is just the ability of students to be active consumers of their learning. I would love for them to take away something that is memorable, that feels unique. And so PsycLearn frees me up as an instructor, that I don’t have to go through every bit of background information in the way I use it in class, I can assume they’ve had things. Let’s spend time on things that are interesting. Let’s dive deeper into things. Let’s do more exercises for things that are challenging. And so the students get to think about and apply new ways of learning and engaging the material from the very first moment that they encounter the content.

Alan: Yeah, for me, it’s really about the inclusion of metacognition, I think it’s really helpful to have students be able to evaluate their own learning. I think this makes them do better on exams, and I think it makes them more successful in their entire college career. It’s also an important part of my own teaching. I have a lot of activities in class where students can answer and identify how well they know what they’re supposed to know. And what I often find is that students study, but they don’t know how much to study, they don’t know: “Do I know it as well as I need to know it.” And this product gives them a lot of feedback, really helps them understand if they understand the material as well as they need to. And PsycLearn really can do that in ways that regular textbooks cannot.

Linda: I would say that my excitement really comes from the big picture, the opportunity for us to deliver on these experiences that Anna and Alan are describing. But we’re going to be able to do that across the foundational courses in the curriculum. And so that larger picture future horizon is pretty exciting.

John: Well, you’ve already addressed some of this, at least in terms of PsycLearn, but we always end with the question: “What’s next?”

Linda: Well, we have sort of talked a little bit about future and what’s next and our opportunity to be iterative. The digital platform allows us to be continually rethinking what is the best activity to offer to reinforce or bolster certain concepts. And so that’s certainly something that is ongoing. We can take feedback from students on how well they feel various content is supporting them. We’ll be looking at what we can do to encourage a greater level and engagement with that support material, so that students aren’t feeling like, “Oh, that’s extra, I don’t really need to go there.” Indeed, it’s extra. But it’s really vital, especially for students who might sort of marginally understand. And so the opportunity for us to deliver a reframe of a concept can make all the difference at that tipping point.

Anna: Yeah, and I think what Linda had said about using the support material is really important to us. So we’re really interested in applying these learning science principles that Alan had talked about to say, “Okay, what is the best way to get students to engage with this.” We want them to see that cool stuff we’re putting on those pages, if they feel like they could benefit of it. So we want to reach the students that need that support. So I think just diving in a little deeper and figuring out the best ways to reach them, to get them to say, “Okay, yeah, I could use a little practice.” Or “I could use another example,” snd really assess that learning of their own. It’s a challenge. It’s a challenge we have constantly, especially when students are new at something, they tend to think, “Oh, yeah, I got this. I know it all.” And so getting better at that metacognition, I think, is really exciting to us.

Alan: Yeah, and I think for us, it has a lot to do with making a better and better product. I think about what it was like the first time I taught a course. And I equate that to the first iteration of a PsycLearn product. And then I think about what was it like the fifth time I taught that course or the 10th time I taught it, and how richer and deeper it was. And I think the same thing will happen with PsycLearn. We’re going to keep on creating a product that gets richer, more inclusive, and more likely to help students succeed.

John: I was really excited to hear about PsycLearn, and everything you’ve talked about makes me even more excited about the opportunity for this and I hope that other disciplines will start working on similar materials for their basic courses, because the benefits from this have been well established in terms of improving student learning and reducing some of the equity gaps. Thank you.

Linda: Thank you.

Alan: Thank you.

Anna: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

219. Rigor

In academia, the term “rigor” is often code for gatekeeping and exclusion. In this episode, Jordynn Jack and Viji Sathy join us to discuss ways of creating challenging courses while providing the support and structure necessary for student success.

Jordynn is a Professor of English and Comparative Literature at the University of North Carolina at Chapel Hill. She is the author of three books and numerous articles that focus on the rhetorics of science, technology, and gender in a variety of contexts. She is also the Director of the Health and Humanities Lab at UNC-Chapel Hill. Viji is a Professor of the Practice in the Department of Psychology and Neuroscience, the Director of the Townsend Program for Education Research, and the Director of the Academic Leadership Program at the Institute for Arts & Humanities, also at UNC-Chapel Hill. Viji is a national expert on inclusive teaching and is a co-author (with Kelly Hogan) of a forthcoming book on inclusive teaching which will be part of the West Virginia University Press series on teaching and learning, edited by Jim Lang and Michelle Miller.

Shownotes

  • Jack, Jordynn and Viji Sathy (2021). “It’s Time to Cancel the Word ‘Rigor.’” The Chronicle of Higher Education. September 24.
  • Nilson, L. B. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Object Lessons – The Atlantic
  • Hogan, Kelly A, and Sathy, Viji (2020). “Optimizing Student Learning and Inclusion in Quantitative Courses.” in Rodgers, Joseph Lee, ed. (2020). Teaching Statistics and Quantitative Methods in the 21st Century. Routledge.

Transcript

John: In academia, the term “rigor” is often code for gatekeeping and exclusion. In this episode, we examine ways of creating challenging courses while providing the support and structure necessary for student success.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Our guests today are Jordynn Jack and Viji Sathy. Jordynn is a Professor of English and Comparative Literature at the University of North Carolina at Chapel Hill. She is the author of three books and numerous articles that focus on the rhetorics of science, technology, and gender in a variety of contexts. She is also the Director of the Health and Humanities Lab at UNC-Chapel Hill. Viji is a Professor of the Practice in the Department of Psychology and Neuroscience, the Director of the Townsend Program for Education Research, and the Director of the Academic Leadership Program at the Institute for Arts & Humanities, also at UNC-Chapel Hill. Viji is a national expert on inclusive teaching and is a co-author (with Kelly Hogan) of a forthcoming book on inclusive teaching which will be part of the West Virginia University Press series on teaching and learning, edited by Jim Lang and Michelle Miller.Welcome Jordynn, and welcome back Viji.

Viji: Thank you.

Jordynn: Thanks. Nice to be here.

Rebecca: Today’s teas are… Jordynn, are you drinking tea?

Jordynn: I’m drinking coffee.

Rebecca: Ah coffee… rebel. [LAUGHTER]

John: It’s one of the most popular teas on this podcast. [LAUGHTER]

Rebecca: How about you, Viji?

Viji: Well in honor of the podcast, I decided to actually make tea. I had a birthday recently, and someone gave me a thing called “Chai Box” and it had teas that you can actually boil and strain and all of that. So I thought I would go through that process and have a nice hot cup of tea with you.

Rebecca: Sounds wonderful.

John: It does.

Rebecca: I wish we were in the same space so I could try some. [LAUGHTER]

John: And I am drinking Tea Forté Black Currant tea.

Rebecca: And I have a black blend called East Frisian.

John: One of your new favorites.

Rebecca: It is, yeah.

John: We’ve invited you here to talk about an article you co-authored earlier this semester in the Chronicle entitled “It’s Time to Cancel the Word ‘Rigor.’” And you raise a number of arguments about the common use of that term rigor in academics. Why is a focus on rigor so harmful?

Jordynn: Well if I can speak, I guess, to the humanities side of the question, I think the term rigor often appears when we have conversations that also entail curriculum, how to uphold a standard of mythical proportions, I guess I would say, of what students used to learn and how they used to write and how they used to perform in class that goes back centuries, honestly. So the complaints about students not meeting some standard are the reason why first-year writing exists. The idea that incoming students at Harvard weren’t writing well enough is why we have first-year writing. And that goes back over a century. But I think in my discipline particularly, there tends to be a concern, mostly about the range of students that now attend university. And I think that’s one of the root causes of this discourse. The idea that… I don’t think anyone wants to say it explicitly, but people seem to think that by admitting more students that have a wider range of backgrounds, we’ve somehow lost some standard that used to be there. And even though we know that this is a myth, it still persists. Often it’s coded language for: “some students don’t belong and others do.”

Viji: Yeah, and I guess I would add, it’s a similar idea in the sciences, I think. In our case, and as we wrote about, I think of this as… these are obstacles for students. And that’s not what we want to be as educators: obstacles. And so I’m really trying to help people see that this kind of language isn’t helpful to students, it isn’t helpful to faculty. It is code, like Jordan talks about, that really privileges a traditional view of education. And that’s not where we need to be right now. We need to be in a place where we support our students, the students we have in our classroom, not the ones that we aspire to have or think we have or any of those kinds of things. It’s actually: Who’s in our classroom, and how do we best serve them? And I think for a lot of people, they just haven’t really examined the notion of rigor and where that comes from.

Rebecca: One of the things that I was thinking about when I was reading your article was also how much this language also appears before they’re our students—when they’re prospective students or are part of the admissions process as well—that their application doesn’t show enough “rigor” or their prior education isn’t “rigorous” enough, to almost prevent certain kinds of students from being our students in the first place, which is really disturbing on so many levels. And we often hear faculty using this language of rigor when they’re complaining to one another about the scaffolding or the support that different students may need in their classes. How do we respond to our colleagues across the university, faculty as well as administrators and staff, who may be using this language?

Viji: I think a big part of it is to not shy away from saying, “We absolutely want to have a challenging course, high standards.” There’s other versions of language that indicate that you want to really push your students, in a healthy way, towards understanding the material with support from you, not as their adversary. And so I think that’s really what we want to encourage people to think about… that this isn’t an obstacle course to build for our students and see who makes it to the other side, it really is a coaching process. It’s a process where, when they succeed, we’ve succeeded, or vice versa. Really, we’re thinking about this as a shared endeavor. And the more students who succeed, the better that reflects our ability, or our perception of how we’re teaching. I never want to be seen as somebody who keeps students out, and if only a small number of students actually meet the learning objectives, that’s a poor reflection on me, not on them. And so I think that’s really what we want to do… is really flip the idea that this is not about you putting up obstacles, but rather you constructing devices, assignments, grading, all of those kinds of things that actually support your students in their learning.

Jordynn: I think to the admissions question, at least in our institution, we’re a state university, we have a mission to serve the state. And I think that’s kind of a persuasive resource we have at our disposal is that we admit students from every county, in our case in North Carolina. We have programs that encourage and provide support for students who are first-gen, we have programs in place to allow all students to succeed or to encourage all students to succeed. And I think sometimes people perhaps assume that we’re in a different role than that, like somehow, our mission is instead to be this super elite institution where we’re only admitting the best and only the best will survive the obstacle course. We’re not Hogwarts. [LAUGHTER] We’re a state institution that’s here to serve the population of students that we have. And I think, for a lot of us, appealing to that mission can be an effective way to counter the idea that we’re instead trying to uphold this elite standard.

Viji: Jordynn’s point is really a poignant one in that this is a source of pride for our campus that we admit a large number of in-state students, and our university is the flagship university in our state. So for both students and faculty, we do think of ourselves as top, we’ve got top students and top instruction happening. That’s why I think we come from this perspective of…  How do we serve these top students who are very diverse in their needs and in their experiences as they come to us? And that we take that on as a challenge, because that’s our job, is to serve these students.

John: One of the challenges that many new faculty have when they come in is they do want to use more inclusive teaching techniques. But some of the people who are evaluating them critique them because they are using inclusive-teaching practices, and they are having more students being successful, which tends to be reflected in higher grades. What can we do to help those colleagues when they’re faced with barriers in those who evaluate them?

Viji: Well, I can take a stab at it, and Jordynn you want to add if you’d like. I think one thing is that we could better educate everyone about the process and what we mean to have an inclusive classroom, to have high standards but help your students meet those standards. And I think the proof is in the pudding. I jokingly tell my students every semester, “If 100% of you make A s, I will be so happy, I will be so happy. If you meet all of the requirements and you do well in the course, that makes me really happy and I’ll retire,” that’s what I tell them, “I’ll retire when that day comes.” But I think just thinking through… How do we show our colleagues that we are asking our students to do really difficult things, and they’re meeting that challenge? That’s what we want to educate our peers about, not the grade distribution, but rather… What did the students do to earn that grade? And how did you scaffold the learning experience so that they were able to do that? And I think when we showed those
learning outcomes and the actual student work, that’s when I think we can really have very meaningful and productive conversations about how to make this happen in more classes, or how to actually highlight this kind of instruction which should be rewarded, not punished.

Jordynn: Yeah. One thing that we learned in our department when we did an assessment of student learning outcomes, was that in courses where there’s more structure, students produced better writing. And those courses happen to be first-year writing classes where each project is introduced, scaffolded with various milestones that students complete. There’s a clear sense of what the assignment parameters are beyond, “Write an essay about this topic.” And we found that students actually produced stronger arguments in those assignments and were better able to marshal evidence to support their arguments. And it wasn’t surprising to me, but it made an argument to the department that this approach is effective. And ultimately, in the report that we wrote up based on this required assessment cycle, we argued that the literature courses needed to adopt some of the pedagogy from the writing classes that was producing these effects. And so I think teaching evaluations might be another way to show that, although that’s more subjective. But for better or for worse, departments are required to have some kind of assessment of student learning outcomes. And I think that’s one place where some of these innovations in teaching can be demonstrated in terms of their usefulness.

Rebecca: We started talking a little bit about what these inclusive practices might be to move in this direction. Can we talk a little bit about expectations and what that might look like, maybe from the start of a syllabus or the course structure, and then maybe move into a couple of details?

Jordynn: I think in the article we kind of took a humanities/science perspective and that’s not a clear split. But I think overall, I’m attracted to approaches that make clear the expectations for students, such as specs grading, which is something I’ve gotten interested in lately, where you lay out, “Here’s what you need to do to get an A in this class.” And there’s different approaches to that, but basically for each unit or for each component of the class, “Here’s what the expectations are.” And then to model, in my case, “Here’s what an effective example of this writing assignment looks like.” And we look at them in class and discuss them so that it’s not this abstract idea, like, “Oh, there was an A paper that exists out there, but I have no idea what it is.” So for each component of the class to be really intentional about making the expectations clear, and then providing structure to support students meeting those expectations.

Viji: Yeah, I think that transparency part is key. You don’t want students to have to guess what is the ideal outcome. And that’s when we know people will make assumptions that are incorrect, and may be hard to come back from. So Jordynn’s example of providing a model is ideal because students can see that model. I’m guessing it’s actually an anonymized student paper, that’s something that is actually like a student product that they can critique, to see examples of that kind of peer work that happens. But it’s really helpful to students to see an example. The criticism I hear sometimes about those kinds of approaches is that it takes the creativity out of what a student might produce, because they may be trying to tailor it to the model that they’ve been provided. And in that case, I think… Only in the examples you provide, right? So you could provide a range of examples that meet the criteria, and students can see the leeway that exists, and encourage them to be creative in those endeavors. But not providing anything… I don’t know if you’ve been watching The Great British Baking Show, but there’s a thing called a “technical challenge” in the show. And it’s a piece where they have some instructions for the bakers and the ingredients, and they’re required to produce this thing. And it’s often something they’ve never even laid eyes on. But the instructions, if you see them, sometimes they show you and it’s like two steps, like, “Make the dough. Decorate it.” [LAUGHTER] And you see them really struggle, because they’ve never even laid eyes on this. They don’t even know, they have to guess based on the materials that are available, “Is this something that I should roll up?” But there’s all kinds of things that they have to guess. And remarkably, because they’re who they are, they figure it out. But why are we making our students do this kind of exercise where they have to guess what the right thing is when we actually have in our mind examples of what the right thing is and that we can provide to them, that gives that structure that Jordynn’s talking about? Structure is so important for inclusive teaching, that we don’t let people try to figure it out, that we actually give them some guidance about how to do things so that they feel confident that they can go into an assignment, understanding what might be required of them, that they have expectations about, say, the time commitment involved in an assignment, how they’re going to be evaluated in that assignment. There should be very little guesswork on the student’s part. Yes, they may not exactly know how you have applied the rubric to their paper, but they have a rubric to look at to sort of check off, “Did I do this, did I include these things, do I feel like I’ve sufficiently provided evidence for certain things…” These are kinds of waypoints that students really benefit from. And I think it produces a higher-quality product from the get-go when you give this kind of structure to students, and then it keeps your grumpiness to a minimum. And I’ve done this and I speak from experience when I’ve given students very minimal structure. And I see the variety of interpretation for an assignment and I think, “Oh, gosh, I should have been more specific about what I wanted.” And then it just keeps everybody happier. You can provide those kinds of guidelines without hampering their creativity. And, in fact, really producing something that is more along the lines of the learning experience you hope for them to have out of that assignment.

Jordynn: I would just also add, at least in my courses, I try to design a realistic problem situation that students are in so that it’s not just me dictating what the expectations are but discovering them together. So for instance, the course I have coming up is a History of Writing class. And there’s a series of essays published in The Atlantic called Object Lessons where people take a commonplace item and write about its history, and often they’re writing technologies. So their assignment in that class might be, “You’re going to write one of these object lessons essays.” So we look at some examples. There’s a history of these writing desks that women used in the 19th century where they would compose their correspondence, for instance. So, we read a few examples, and then we generate the criteria together. So what makes a good object lessons essay in The Atlantic? What’s goi ng to be likely to get published in this series? So then we develop the rubric together, we develop the specifications for what a good essay will look like, and then they’ll see, “Okay, well, this author approached it in this way, this author approached it in a different way.” So there’s leeway there. They all kind of fit a general sub-genre of writing. But it’s not like I’m developing these arbitrary requirements, like, “It has to be this long, and it has to use this citation format.” But together, we’re developing a sense of how to effectively address this situation. And so I think that addresses some of the concerns sometimes that highly structured approaches are too one-way, like, the instructor is just determining what could be still relatively arbitrary set of criteria and making them clear to students. That’s better than nothing. But it can still be ineffective for students if you’re arbitrarily deciding, like, “It has to be exactly 500 words. And I’m going to make it clear that every citation has to be exactly correct. And you can’t have any misspelled words or you’ll get a zero.” That approach is transparent, but it still might not be inclusive.

Viji: What’s nice about that is that it conveys the ambiguity sometimes of the work and how it isn’t always cut and dried, even from the perspective of the evaluator. And so it’s really helpful to students to kind of peel back what’s happening and seeing, “Oh yeah, this example resonates so much with an example I use in my class where students have to craft an interpretation around the standard deviation.” And we talk through different examples, and they see why some are more limiting than others. And it really does open up this discussion of what makes a better interpretation of a standard deviation and why.

John: One of the things you point out is that a lack of structure is not terribly equitable, because our students come in with very different preparations, depending on whether they’re a first-generation or a continuing-generation student and the type of preparation they had in their local school districts, which varies a lot based on the average income within their community. So adding structure will not only help all students, but it will particularly help those students, I think, who come in less prepared to play that game of higher ed, who come in with less clear expectations about what we expect of our students. What are some other examples of ways in which we can build in more support for those students who come in with less preparation for the hidden curriculum of higher education?

Viji: I’m going to start by… something I’m heavily involved in that I love both the idea and the implementation of it at this stage. We are in the process of incorporating a new general education curriculum on our campus. And I was part of the coordinating committee that helped think through how this curriculum might look. And now we’re in this phase of rolling out a course that does exactly this. It’s a first-year thriving course, and it’s about thriving in college. And the goal is for students to take this course in their first year, it could be spring or fall, but to really unhide that hidden curriculum of college, and particularly at a research university like ours, where students may not be familiar with the kinds of resources that are available to them, how to navigate the research university, what exists for them to really seize as part of their experiences in college. And a big portion of this is the science of learning. Because many of our students have been through many different types of classes, but don’t actually have a lot of knowledge about effective study strategies, good note taking… there’s all kinds of things that people have just been winging it. And because they are top students in our state, their method has worked for them up to this point. But oftentimes, it falters in college because it is a bit different. And some students are better equipped to handle the kinds of things we’re asking them to do in college, because of their preparation, or the methods that they’ve chosen have really served them well. So we don’t want people to bump into good practices, we would like to be able to educate them about this from the start. And that’s one thing that I’m so excited to see our students be able to take advantage of, and it will be for all first-year students, it’s a required course. And it will really help, I think, our students not only see good practices, but also point them to places they can go in the future, like using our Learning Center to get ongoing academic coaching, to really think about all of the libraries that we have and the services they offer, to really serve them well throughout their time at UNC.

Jordynn: Yeah, this makes me think about also the research on transfer, which suggests that it’s often really difficult for students to apply something abstract to a new situation. So let’s say they take this course, they learn about study techniques, then they’re in your history class, for whatever reason, they might not necessarily think back to that first-year course and say, “Oh yeah, I learned this technique.” So we have to prime them and cue them to draw on what they’ve learned previously, in order for it to transfer. So I was just working on my course design for next semester. And we’re doing a pilot of Canvas, the learning system. And I noticed that the modules that have been pre-designed have these troubleshooting sections that are for technology. But listening to this makes me think I need to repurpose those for troubleshooting the assignment. Like, I was reading through one of the readings that I assigned and it uses terms like “autochthonous.” And reading it, I was like, “I don’t know if my students are going to know this.” But will they know, “Oh, you can look up terms in a dictionary?” Which seems
obvious, but it’s not something you remember to do when you’re reading a difficult text. So troubleshooting, like, “If you’re having trouble understanding this text, here’s a good reference dictionary that might have terms related to our class.” Or, “How should you study for this test that’s coming up? Here’s a reminder about this study technique that you probably learned about in your first-year thrive class, and here’s the link to the Learning Center handout about it.” Like building in those kinds of supports the way we would for technology support. So the metaphor of troubleshooting, I think, is really helpful and something instructors should be conscious of. The same thing happens in writing all the time, like, “Well, they obviously learned how to cite sources in first-year writing.” Well they did, but you’re asking students to cite in Chicago, and they haven’t seen Chicago before. They’re not going to recognize that what they learned about MLA or APA transfers to learning about Chicago. And they won’t know how to do it unless you give them some resources and some support. You can’t just have this expectation that something they’ve learned before will automatically pop into their brain when they’re in the moment of working on something.

Rebecca: Jordynn, what you’re saying is reminding me of different experiences I’ve had with students when they get panicky about something they’ve not seen before, like, “Ah! I don’t know what this is, I’ve never seen it before!” And then as soon as you start panicking, all the things you know somehow have disappeared out of your brain, they’re no longer accessible. [LAUGHTER] So having those little reminders about those basic things is really helpful. Because when you are really under a high-stress situation, which you might have self-imposed, potentially [LAUGHTER], you still have that place to go back and re-center and go, “Oh right, right. I know that.” But those little cues can be useful. I know that in conversations I’ve had with students who hit that moment of like, “But I don’t know how to do this!” Yes you do, let’s walk through things that you do know how to do. And all of a sudden, you can just see it on their face like, “Oh right, okay.”

Viji: Yeah. And the reminders, they’re more than reminders, it’s actually reinforcing the learning, which we know is really helpful. We know things like recalling previous information is helpful for really storing it long-term. So you’re really just actively producing that retrieval over and over again, by bringing it back in other classes. But it also conveys who you are as an instructor and how you’re supporting your students. And they read between the lines sometimes when you provide these kinds of things. They come to you, and they ask about other things that you could offer advice about. It really opens the door, and it gives them permission to say, “Oh yeah, I don’t actually have to know all the words, I can look them up. That is something that they’re not assuming that I know all of these words, because they’ve told me how to use a dictionary when I’m reading the text.” So it really does give students permission to say, “It’s okay if I don’t feel 100% confident in doing this, my instructor expects that not all of us are going to know the terms.” So these are the ways in which we support our students in our communications that really help them see that we want them to succeed.

Rebecca: Another example of that is also modeling that, as a faculty member, when things come up. I teach a lot of classes where we’re doing code stuff, and I obviously am not a walking code dictionary, I can’t just recall everything. And so I often get asked questions that I can’t quite remember, I know enough to be able to look it up. And so I’ll demonstrate, “Here’s how I’m going to look this up, these are the search terms I would use.” And I have found that students are shocked the first time I do that, like “Wait, she doesn’t know everything?” But also I think they find it helpful, because they’ve said things like, “Oh, I wouldn’t have thought to look like that, or I wouldn’t have gone there.” And then by the end of the semester, I had students this semester who’ve been telling me, “Oh, I have that open all the time while I’m working on my projects now.” Good, good.

Viji: Rebecca, that’s a great point, and I think I’d be remiss if I didn’t just say that not all faculty members can  do that. We have colleagues who, if they falter in what they know or don’t know, that students will really judge them harshly for that. And so I think it’s amazing when we can do it, but realizing that some faculty members are not going to have the ability to do that very comfortably in the classroom.

Rebecca: Definitely, definitely.

Jordynn: Yeah, I’ve been called wishy-washy on evaluations before for what I felt was being inclusive in terms of negotiating with students… What are the parameters of this assignment? What should be the length for this? Or when do you want it to be due? I think sometimes students want more authoritativeness than sometimes I feel is suitable or appropriate. And so that’s something I’m constantly negotiating. And I think if I were in a different body, you know, maybe as a senior male professor in my department, maybe that wouldn’t read the same way. But it has happened on occasion, where I think some students also just feel comfortable with, “Here’s what you need to do.” If you’re used to the obstacle course, you just want to know what the hoops are. “Just tell me what to do and I’ll do it.” And so taking a step back from that approach is uncomfortable for some students and also can reflect on how the instructor is evaluated by those students.

John: To create this inclusive environment, what are some things faculty should avoid doing in their classes?

Jordynn: Well, I think for me, assuming students should know X is, to me, a problem. So of course, there’s prerequisites for some classes and things that you would assume students already know. But, at least in my department, students don’t proceed through the curriculum the way we think they do. So we have this idealized path, but because of the difficulties they have in scheduling courses, or just the different paths students take, like, maybe they decide later on to become an English major, and they have to go back and take some of the earlier courses. We can’t just assume, “Oh, they must have learned this in X class.” So that’s for one, something that can really alienate students, when you think, like, “Well, they must have read X,” or, “They must understand this concept,” or, “They must know how to do this thing.” And to me, my principle is that I don’t evaluate anything I haven’t taught. So sometimes I have to be reflective about that, like… Am I trying to evaluate something that I haven’t taught them implicitly, because I have these kinds of assumptions? But for me, that’s a big one. It comes across in the attitude of the instructor sometimes too, like, “Well, obviously, you all know this,” like, “If you don’t know this, why are you even in this class?” I’ve been in situations like that, too. [LAUGHTER] It’s like when you go into a fitness class and you need to get back in shape. And they’re like, “Drop and give me 20 push-ups.” And I’m like, “Well, I can’t do a push-up right now. So I guess I’ll just walk out the door.” So having these rigid expectations about what they already know, before you’ve even taught them anything.

John: Now in some classes, particularly in STEM disciplines, there is some assumption of prior knowledge, but students don’t always have that knowledge when they come in. Are there ways we can support those students, while still allowing those students and the rest of the class to advance to the higher-level materials?

Viji: Yeah, I think you’re bringing up a great point, John. And Jordynn has mentioned this now a number of ways and times, that just because you do something once, doesn’t necessarily mean you’ve learned it. And that’s what we keep saying, it’s true of learning. There are very few things where you do it one time, and you’ve figured it out for the rest of your life. And I think we need to remember that as educators. And the way that I remind myself of this is every year or two years or whatever, I pick up something that I’ve never done before, that is hard for me. Right now, it’s playing the guitar. And I take a break from it, and then I come back and I’m like, “I don’t remember how to do that chord, again, I have to re-remember the fingering.” And then it’s just important to remember that they’re doing this not just for your class, but they’re doing this across multiple classes across multiple terms. They may have had that course three semesters ago, rather than just last semester. So of course, they’re going to have differing levels of recall about the information. Plus they’re not walking encyclopedias, they don’t remember every fact that was ever told to them. Neither would we. I mean, if I think back to my college days, and you tell me to recite a fact I learned about a particular course, I’d be hard-pressed to come up with some ideas. And I hate to say this, but I barely remember some of the instructors’ names, and I saw them every day, or every other day, for a whole semester. So we have to have realistic expectations about what can be recalled, versus… what can we give them so that they can go back and reflect on, pull back that information, to remind them of it over and over again, so that it does become a bit more cemented. And a big part of that is just recognizing the unrealistic expectation, that just because they saw it once before, that they will be able to hit the ground running with it in your course. And that you do need to provide some of these ramps on to the material. And that can include sometimes a period of time at the beginning of the semester where everybody’s just practicing some of the foundational skills that you might presume that they’ve had, but again, might be rusty, and they all need some practice with it. And I think peer instruction is a great example of where you can ask people who feel less rusty to help those in the classroom, or have students who’ve recently completed the course come back and help students. And that is another way to sort of keep that recurring practice going beyond the semester of learning that content.

Jordynn: Yeah, I agree. My difficult thing that I picked up recently was I took a weaving class. And it was a six-week class where I learned from start to finish how to make this sampler of different weaving techniques. And from the very beginning where you have to warp the loom, to using the shuttle, I don’t think I could do any of it now. But I think if I went back to the class and retook it and the instructor showed me again, “Here’s how we did the warp.” I’d be like, “Oh yeah, okay, I remember.” Some of it would come back to me, some of the techniques are embodied almost, and I would be able to pick it up again. So it’s the same thing. It’s like the baking example, like, “Make the dough.” Well that’s really difficult, but if you reminded me of the steps, I’d be able to make the dough if I’m a baker. So I think, similarly, it goes back to providing scaffolding and supports for whatever the assignments are, so that we can practice and then we can go through the steps again. And over time, if you do repeat that type of task, eventually there’s going to be some development. But you can’t just put the task out there and expect people to remember how to do it.

Viji: John, you started with a question about, “What should we stop doing?” And I have to say this just to get it out there, people should stop grading on a curve. People should stop assuming that a certain percentage of students earn an A, earn a B, C, D… whatever it is, however you’ve designed your course, that is not the way to, quote/unquote, “sort” your students, to have some predetermined thresholds for grades. There are a lot of really poor reasons for doing that, aside from the sort of quantitative side of things. Like people vary from semester to semester, and you could potentially have a really outstanding group one semester and still force people to earn bad grades. To the more principled ideas that you should actually have standards that you want students to meet, and evaluate students on those standards, and then decide, “Is this A-level work, B-level work?” And if you’re doing this collaboratively with students, that they really think through what that looks like for them, too, rather than just saying, “I already know what my grade distribution will look like, because I’ve designed it that way at the
start.”

Jordynn: I would say to stop nitpicking, too, at least in my field. I had a student in my class say, “Well, I got a B on this paper because I didn’t put the date in the right format that the instructor wanted.” [LAUGHTER] I was like, “Okay, she took a letter grade off for a date format?” Like, I get that maybe there’s some cases where really specific things are really, really important. But when it kind of seems arbitrary I think that’s another way of just saying, like, “Well, I can’t have all As. So I’m going to find relatively minor things to critique students on that will help me to distribute the grades differently. And if it has to be the format of the date, or a period in a citation, or something like that.” Personally, that’s not what I want to be spending my time on, and I don’t think that’s the most productive use of my feedback to students. And it also ends up having an effect that’s pretty uninclusive.

Viji: Yeah. And your comment reminds me, too, of some discussions we’ve been having about flexibility and offering more flexibility to students, particularly now. And we have some resistance. And I’m speaking “we” collectively as educators, about, like, upholding deadlines for things. And when I say, “But where in your syllabus is this one of your outcomes, to hold people to deadlines?” Because that’s often the underlying rationale is, “When are they going to learn how to turn things in on time? [LAUGHTER] In the real world, you have to turn things in on time.” And I want to say, “Really? I mean have you never asked for an extension on something and been given grace on it and felt so relieved about it?” And then secondly, that doesn’t sound like a learning outcome. That sounds like maybe a skill, you’re trying to help them with professionalism, and maybe part of professionalism is completing work in a timely manner. But I think a lot of times, it’s really just a matter of convenience for all involved to turn something in by a date. And so really examining… What are the things you think you’re really asking or upholding for your students? And why that is. And if it truly is, like, “I really want to help students stay on task and get things turned in in a timely way to get feedback,” then explain to your students why deadlines exist and why you have those kinds of approaches in your class.

Jordynn: Yeah, the number of professors who write their conference papers on the plane or the night before in the hotel room. Or if you’ve ever done an edited collection, you’re lucky if 30% of people turn in a draft on time. How often are deadlines extended because no one’s submitted the paper or the abstract for the CFP? Like, we’re the worst at upholding our own deadlines, but then we try to turn around and enforce them on students. And it’s really nonsensical.

John: There are cases, though, where deadlines could be useful. That if students are going to be doing things later in the class that depends on the development of skills that they need to have first. How do you deal with cases where it’s important that students master skills to be able to move on to higher levels with the class? Or they’re working in group activities where all the students need to have developed skills to move forward, and some of the students are lagging behind?

Jordynn: Well, I give the grace period for all of my assignments where there’s a window where the assignment is open and that’s the suggested deadline. Like, “We do have to move on to the next unit, so here’s a five-day window, and it opens this day.” And then I have a clause like, “Okay, if you’re really struggling, you’re having something going on, just come meet with me and we’ll work something out.” So there’s two layers, like, there’s flexibility that applies to everyone and then there’s, “Okay, something major is going on in my life.” And increasingly, that’s happening a lot. And one of the benefits, I think, of our remote learning switch is that it’s kind of changed my whole pedagogy where I have more things built into the course management system, whereas I used to just use it in a really minimal way. So now I have modules that you can work through on a more self-paced way. And it’s better if you’re doing it in the same timeline as the rest of the class, because we are using class time to talk about elements of that. But I think, in the long run, having built those materials will allow students to proceed in those extreme cases where there was something else going on in their life, and they just can’t be in class. I think it’s not ideal for students to take an incomplete and then try to finish a course later or make up the work in some way. So anything you can do to avoid that outcome, even if it means creating an alternative pathway, I think is worth doing.

John: That can work really well in smaller classes, but how does that scale when you have hundreds of students in the class? Just as an example, one thing I used to allow students to do, this was way back when I first started teaching large classes, I had weekly exams and so forth and let students replace those by submitting a video project. And I let them do that up through the end of the semester. And then there was one semester where I had about 120 videos to watch [LAUGHTER] in the last week or so of the semester, and that became a little bit overwhelming. Now, perhaps there were some ways of doing that differently, but it’s a lot easier to do that in smaller classes. How can you maintain that sort of flexibility in a larger class?

Viji: I think that is the million dollar question a lot of people are asking themselves. It’s almost like a universal design-for-learning type of question, where it’s like… How do we do this so that it’s accessible to lots of people? And I don’t necessarily need to know rationale, reasons for why something might need to be late. I think the grace period is a great example where just everybody gets it. Like, I don’t need to know why you need the grace period this time, you just get it. I’ve also heard of the notion of “oops tokens.” Life comes up, and you just can’t turn in the assignment or take the quiz, and that’s okay. I’ve got a few of those built into the semester. And the way that I implement that in my learning management system is to have a column that is just those passes that we give to students. And so we know we’ve kept track of the passes that they need, like, they contacted me to say they can’t turn in assignment two, great, that’s out of, let’s say there’s five oops tokens, that’s a one for them in that column for that semester. And then we just keep track of it that way. I think there are some ways, but unfortunately, I don’t think our learning management systems, our technology, has really incorporated flexibility in its approaches. And that’s where I think I’m personally very interested in how technology can help us be more flexible in keeping track of things like this. I mean, it is helpful to say, like, “Okay, best due by a due date, that is the ideal timeline to get good feedback to be able to move forward.” But then there’s also the, “It is not going to be late, you can absolutely turn it in, here’s another deadline. And then maybe there’s just one that’s a wide open, like, “By the end of the semester, I just need to see something from you.” And so then students aren’t panicking, that they’ve got a “late” marked by their paper because it got turned in anyway. So really just thinking through how to leverage some of our technology to do that. But I say that saying, I don’t think our technology is ready for flexibility yet, it isn’t where it needs to be. It can do things like drop a grade in a grade set or something like that that can also be helpful. But I would love for it to be more flexible in the way that it can calculate grades, like even offering different grading schemas that students can choose. And then it actually projects those grades for students so they’re not having to do the calculations themselves and second guessing where they need to be on something to be able to earn a certain grade. So I think if anybody out there works at any of these learning management sites, I want to talk to you about how we need to be better about really using good pedagogical approaches, and baking them into the systems that we have. I think this is the biggest challenge for faculty. Their heart is in the right place, oftentimes they want to do these kinds of things, but they can’t figure out how to do it at scale. So if we can get technology to help us, then I think we could actually capture a lot of this challenge.

Jordynn: Yeah, I think the course management system has a baked-in pedagogy that’s not always the most progressive. And it’s often content delivery, a term that I hate, with some kind of quiz, and then they add assignments in. But sometimes the default settings revert to this kind of model. So I noticed in this Canvas pilot that we’re doing, the collaborate tool is, for whatever reason, either disabled or really opaque in terms of how to use it. But that’s the number one reason why I wanted to try something different, was to be able to offer built-in ways for students to collaborate in a Google Doc or to use Google Drive without sending them to some totally different site, which gets really confusing to students. And the built-in assumption is that that’s not the primary thing that instructor wants to use this for. Yeah, I mean, I don’t know to what extent they actually talk to instructors who are trying to build in different approaches to teaching, but it seems instead there’s this kind of default, like, you’re just going to be uploading notes and videos for students to watch and then answer questions about and that’s your pedagogy. So yeah, it’s pretty problematic in that way. Figuring out contract grading, also, which is an approach that I often use, has been really difficult for me, which is a totally different approach where I’m not marking each thing and giving students grades but that they’re trying to meet specifications for what it means to earn an A in that class. And so something more holistic like that is just really hard to enact.

Rebecca: So we always wrap up by asking the really big question, “What’s next?”

Viji: I want to bring it back to this conversation of rigor and say—and you all know this—my colleague, Kelly Hogan, and I have been talking about inclusive teaching for years now. And while we do receive some critical remarks about it, nothing like this article about rigor has actually produced. Somehow we pushed a button that some people really, really didn’t want us to push. And it’s been eye opening to me because it’s very much the idea of inclusive practices. But with this explicit idea that rigor should not be attached, like, we should actually not use that word. So it almost feels like there’s some people out there with an alert for the word rigor. And then when that pops up, they get right into it. It’s been interesting for me to see. This is definitely a hot button issue. And although I think inclusive teaching approaches can help ameliorate some of them, it opens my eyes to the ideas that we have a lot of different people who have different ideas about education, some of them are actually educators, and some of them are not. So thinking about, holistically… What other things are we not challenging that we should be because the underlying assumptions are gatekeeping in the very worst ways by turning people away, which is not what we want to do. And really helping people to see that these practices, these terms, are perpetuating privilege that really, as educators, we should absolutely challenge. I think this is something that we don’t stand for, to say that only those who have could continue to have, and that we really want to level the playing field and help all students succeed. So those are the kinds of things I’m thinking about… What else is out there? I’ve said rigor. What else can I say to make sure that people know that these are not what we want to be doing in our classrooms today?

Jordynn: I guess for me, the elephant in the room is the effects of COVID on generations of students. I have elementary-age kids, and it’s affected them in certain ways. And then obviously, kids who are in high school right now, the students that we have in college who are coming in with two years, three years of disrupted learning. That’s a huge opportunity, I think, as well as a challenge for thinking about inclusivity because the range of effects for different students is so vast. And I think, relatedly, the experience with COVID has brought up questions about standardized testing. And I think that’s another important place to think about inclusivity and to think about the rigor assumptions that shape who gets into college. And the kind of temporary moratorium for some places on using those scores, opens up an opportunity to question how the emphasis on standardized testing has shaped our students’ education, because it starts in kindergarten now. And, you know, we’re holding students to one very specific, relatively narrow standard about what constitutes learning. And I think especially with the effects of COVID, it should become apparent that it’s just unfair to keep doing that when students have been out of class, they’ve been learning virtually, students have lost parents and loved ones. And just sticking to this old model where everyone has to meet this one really narrow standard… we can’t keep going that way with everything that’s happened.

Rebecca: You’ve both brought so many really important topics to the table today. And thank you for writing your article and bringing up the word “rigor.” And I hope you find more [LAUGHTER] that we can start to tackle as a community.

John: In terms of the response you’ve received, this response is really bringing up some critical debates, I think, on a lot of campuses. And, again, as Rebecca said, thank you for writing this. I hope it spurs a lot of productive debates on many campuses.

Jordynn: Thanks for having us.

John: Thanks for joining us.

Viji: Thank you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

218. Blended Learning

Although new to many as a result of the pandemic, blended learning has a long history of effective use. In this episode, Chuck Dziuban and Patsy Moskal join us to discuss how blended learning has been used at the University of Central Florida for the past two decades. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida [UCF] where he has been a faculty member since 1970, teaching research design and statistics. He is also the Founding Director of the university’s Faculty Center for Teaching and Learning. Patsy is the Director of Digital Learning Impact Evaluation, also at the University of Central Florida. Chuck and Patsy are both Online Learning Consortium Fellows and have been doing research on blended learning for quite a while now. They are also two of the editors of the recently released third volume of Blended Learning: Research Perspectives.

Shownotes

Transcript

John: Although new to many as a result of the pandemic, blended learning has a long history of effective use. In this episode, we examine how blended learning has been used at one institution for the past two decades.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Chuck Dziuban and Patsy Moskal. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida [UCF] where he has been a faculty member since 1970, teaching research design and statistics. He is also the Founding Director of the university’s Faculty Center for Teaching and Learning. Patsy is the Director of Digital Learning Impact Evaluation, also at the University of Central Florida. Chuck and Patsy are both Online Learning Consortium Fellows and have been doing research on blended learning for quite a while now. They are also two of the editors of the recently released third volume of Blended Learning: Research Perspectives. Welcome, Chuck and Patsy.

Chuck: Well, thank you, Rebecca. We’re glad to be here.

Patsy: Yeah, thank you very much. So nice to be here talking to you guys.

John: It’s great to see you back, Chuck. And it’s great meeting you, Patsy.

Chuck: Yeah, this is my third time. I’m honored.

John: Our teas today are… Are either of you drinking tea?

Chuck: I am drinking lean green tea from a Christmas mug.

Rebecca: Perfectly in season.

Patsy: Yeah, I’m just drinking a green tea. My usual afternoon drink.

Rebecca: I think I’m accidentally drinking watered-down English afternoon tea. I think I put hot water in my cup without putting a teabag in it. And so it’s very light.

Chuck: You’ve always been innovative, Rebecca.

Rebecca: So it’s hot water. [LAUGHTER]

John: And I am drinking a peppermint spearmint tea blend, as it is late in the day and I had a lot of caffeine earlier.

Rebecca: Miss your blends there, John. Haven’t had one in a while.

John: Well, I’ve got some new fresh tea to make them.

Rebecca: I’ll have to stop over. So we’ve invited you here today to discuss blended learning. First, it might help if we start with the definition of blended learning?

Chuck: The definition of blended learning could take this and several other podcasts. The whole notion of blended learning has sort of evolved by complexity. It is diverse, it is interdependent, it is connected, and it is adaptive. I remember the early days with the Sloan Consortium where Frank Mayadas, who was adamantly opposed to blended learning, had wanted nothing to do with it. And Tony Picciano and I finally convinced him to fund a small summit in Chicago to begin discussing blended learning. And the first task, of course, was to define blended learning, and that was 30 years ago. And here we are, still defining blended learning. Blended learning is emergent, it is changing constantly. It almost defies definition. If you look at the literature, Oliver and Trigwell writing that famous article “Can ‘Blended Learning’ Be Redeemed?” and castigating it because it is not scientific, it has no specific definition and never will. To Rhona Sharpe saying its great strength is the fact that it is undefined and it allows every campus to develop its own model of blended learning. So blended learning definition is a very, very tricky kind of thing. It’s going to be some combination of online and face-to-face education, but it has evolved way past that, as you all know, in terms of doing this. I think basically—and I’ll let Patsy chime in in a minute—it’s become what Susan Leigh Star has called a “boundary object.” Boundary objects are something like critical thinking. If you go to the UCF or Oswego campus and you say to faculty, “Are you in favor of critical thinking?” “Oh yes, we are. Critical thinking is wonderful!” But then you get the group together, you get the faculty consortium together, the Faculty Senate, and they disagree. They fight vehemently over the definition of critical thinking. However, if you go back to their individual constituencies—rhetoric, physics, mathematics—they know precisely what critical thinking is. And that’s very much the way blended learning has evolved. It’s moving, it has changed the way we think about this and it is emergent. It will be emergent for the next decade.

Patsy: Just from an institutional standpoint, we do have a definition of blended learning. We actually refer to it as “mixed mode” on our campus. And it’s a combination of online and face-to-face, but it has to have some reduced seat time. So for instance, if it’s a three-credit-hour course, there may be one hour that is in a physical classroom space, with the remaining two hours being asynchronous online instruction. So, we have a newer modality that actually started in our College of Business to have large enrollment. And we don’t have classrooms large enough to fit even a fraction of the students. So, they looked at a mode that has less than 20% face-to-face. So basically, meet five times in a semester, splitting up a large class into five smaller groups, and they focused on making those active learning. And the remainder of the instruction, again, is online. So for us, it just has to have some reduced seat time capacity.

Chuck: Well, Patsy has given a great definition of logistics associated with blended learning. And then you have the educational implications of blended learning. There are great thinkers about this. Charles Graham has done a great deal of work on the kinds of blends, augmenting blends and supplemental blends and all of those kinds of things. Anders Norberg in Sweden who has done a marvelous paper on looking at blended learning simply from a time perspective… Do you want to be synchronous or asynchronous? So there are many forks to this kind of thing. What do you do with seat time? What do you do with educational philosophy? And as we talk about modality, and I will say now and people probably won’t like me, but course modality is one of the worst predictors of outcomes ever, in terms of doing this. When you look at any kind of predictive model, students don’t care about modality, we’re the ones that seem to be obsessed with it.

John: Could you tell us a little bit more about how the blended learning program at UCF got started?

Chuck: We started this whole thing with online learning. And Patsy and I began to collect data immediately. And the administration said, “Oh, this data is very informative, collect more of it.” So one of the studies we did is we looked at the presence of students who were taking online courses. And lo and behold, we found that the vast majority of them had a campus presence. All of these distance students that we imagined didn’t really exist. They were taking courses from labs, they were taking courses from the library, and they had a presence on campus. So it led to the conversation of… Well, if they’re here, can we somehow use the affordances of both modalities, of the face-to-face and the online learning? And what happened was, we developed, and Patsy’s already mentioned, this notion of, we call it “mixed modality.” We like to think that we invented it, but everyone else likes to think they invented it as well. And we stuck with mixed modality for a very long time, until Sloan began funding blended learning. And then we changed very quickly to blended learning. Patsy, what did I leave out about that?

Patsy: No, I think that’s essentially how we started doing blended learning. And we’ve never had enough classroom space, I’m sure every college is the same. So, part of the reason that we looked at online and blended was to increase access for our students, to help people, provide them with the means to be able to get an education. And it was very appealing for the administrators to look at the concept of blended learning. Again, if you took that three-credit hour and built one-credit hour with face-to-face, well I can fit three of those blended classes in one classroom over the course of the semester. So that’s an appealing use of maximizing the classroom space. And I think our past CIO, Joel Hartman’s, idea was to do that. Ironically, It’s a lot harder than you think because of the way scheduling is done. As you can imagine, departments don’t necessarily coordinate. We all know how it works functionally, sounds great, but when you try to put it into play at a university sometimes there’s things you don’t plan on. But we did have our Rosen School of Hospitality Management within the last, I’d say, five to seven years. And they’re down at our International Drive area because, not surprisingly, that’s where all the hospitality jobs are, so it makes sense. And they had a huge parking issue. So they had not been interested at all in blended learning until they realized they could use it for this exact reason. And then they jumped on the blended learning bandwagon and are now one of our biggest users of that modality. So it’s really interesting how, if you’re thinking outside the box, a lot of this comes to play.

Chuck: That’s an interesting story. There’s a great aphorism by the great philosopher, Yogi Berra, who says, “In theory, there’s no difference between theory and practice, except in practice there is.” And this is what’s happened and the fact is that although the Rosen College began their blended learning initiative out of a not-so-educational perspective, again, this logistic thing. And I will quote that our former CIO, Joel Hartman said, “If you ever think you’re going to make money with this, you’re in the wrong business. It’s not going to happen.” But the other thing has happened, the Rosen College has developed into a marvelous, marvelous educational system using the blended learning model. Maybe not for the right reason, but they have evolved,. and they’re doing things extremely well, because they’re a natural for the combination of hospitality management, face-to-face, and online learning.

John: So blended learning started there mostly because of limited resources and the need to use the resources more efficiently. How did it work? What does the research tell us about how effective blended learning has been compared to other modalities?

Chuck: Well, imagine yourself as a faculty member who has been told, voluntold, that we’re going to do a blended learning course. So now what you’re going to do is you’re going to have part of your course online, part of your course face-to-face. The first time, almost always, a faculty member starts, they start with this thinking: “What in my face-to-face can I offload to the online environment?” And they think about that, and that is usually the beginning of this. And then it begins to emerge when they say, “Ah, maybe this is not the way to think about it. Maybe I need to think about it in terms of which is most effective online, in which students do the best online, in which students excel in the face-to-face environment.” So what happens, John, is there’s an evolutionary development of faculty members, to the fact where they no longer think about this as two modalities. But they think about this as a constructed course with seamless boundaries. And that really happens over several semesters. If you’ve ever done it yourself, teaching the first course in any of these modalities, which we have, it’s a disaster. It just is not the way you think it is. We’re all pretty experienced at this, and we all know when it goes well. And we all know when it doesn’t. And nothing feels better when it goes well, and nothing feels worse than it goes badly. That’s the way it happens, it evolves. Now the outcomes from that is a story that we can talk about later.

Patsy: I think UCF… early on… we were lucky that we got involved with online and blended learning when we had enough resources to be able to do it right. And we had visionary administrators who recognize that it’s important to focus on the faculty development, it’s important to focus on research and try to use research to help inform the process. And so all of those units were put into place early on. And the late Dr. Barbara Truman actually led the Faculty Development Initiative. Dr. Steve Sorg, who’s retired, led the Center for Distributed Learning, and then Chuck led the research arm of our beginning part. So in faculty development, one of the main drivers and we’ve always said, “Don’t let the technology drive the instruction, but let your instruction drive the technology.” So, as part of the faculty development, faculty are encouraged not to just figure out what to put online versus face-to-face. But really think about redesigning your course from the ground up. What are your learning objectives? What are your instructional objectives? What do you hope to do? And then find the best tool… whether it’s online, face to face, active learning, adaptive learning, whatever… find the tools available that you can do well, and go from there. So I think that’s been part of it. And UCF also has a detailed process for quality review for courses. So if you’re teaching online or blended, you are required to go through the faculty development and get certified. But that gets you access to an instructional designer who helps you think through not just the technology, but the pedagogy behind it. You get access to graphic artists and we call them tech rangers who develop all kinds of great tools and games and simulations and all that kind of stuff. For adaptive learning, we actually have instructional designers that focus on personalized adaptive learning. So they know the software and the instructional components of adaptive learning and how to do it well. That’s the big carrot, they get incentivized to come through the program. But I think, once they get in, they actually like some of the stuff they found. So, in redesigning their course to be online and/or blended, they find tools that they then want to incorporate in their face-to-face courses as well. So that’s historically been what we’ve done, what we’ve seen. And it served us well, I think, to maintain quality across the institution.

Chuck: Well, what you see also is, that Patsy didn’t mention, is in our training of faculty, they experienced the blended learning course. They actually are involved in a course that is blended learning. So they understand, as students, what they are confronting in terms of this. So it is, again, evolutionary in terms of a changing process. And I will say that every faculty member who comes in from whatever department thinks that the pedagogy and unique problems associated with their discipline is in fact unique. That what you have to do in rhetoric, what you have to do in physics, is vastly different than any other discipline. And we are special and we are unique. At the end of this course, they realize that there is common ground, mostly all common ground in terms of what constitutes effective instructional things. And they’re able then to look at all of these technologies we present to them and be able to evaluate them, “Is this technology really a solution looking for a problem, or is it something that can actually help me become more effective in my instructional design?” And it is, again, an evolutionary process. And you’ll see a coming together of faculty during this experience. And they don’t all buy it. We’re instructionally design oriented. And I always remember an incident of someone from rhetoric came into my office and threw a copy of Derrida on my desk and said, “What is this instructional design stuff? When you understand deconstructionism, we will talk.” So I read the book and said, “The book deconstructed and disappeared. Now, what do you want to talk about in terms of doing this?” But the point is, you have to respect the values of the discipline. And I say that with tongue-in-cheek, but you absolutely have to respect the value structure of any discipline you’re working with. If you tell them they’re doing it wrong, you’re not going to get far.

Patsy: We also rely on something called “web veterans.” We have our web vets, which are faculty who have already been through the process. And not surprisingly, I think anytime you have a new initiative, there are those people who jump on board, the first people out of the gate who love to test drive stuff. But they become the experts, and they know how to do it well, and they know the pros and cons. And they also add credibility. So faculty listen to other faculty, particularly those within their discipline, or STEM listen to other STEM faculty and so forth. But it’s much more credible than having an instructional designer telling me how to teach when you’re not in my classroom, as Chuck said. So I think it’s really helped. We have a faculty advisory board that really helps direct what we do and think about the evolution as we’re now in a state of evolution, I think, and with COVID, are going to be in a state of evolution. I think that really has helped.

Chuck: Well, we’ve all taught, we’ve all taught for a long time. And there’s a great deal of difference between teaching a few courses, and teaching year after year after year after year over the long haul of this, and we’ve all taught and we’ve all taught semesters. And I think all of us experience this excitement, and anxiety at the beginning of a semester. You say, “I really want to do a good job, and I’m all excited and I’m nervous.” And at the middle of this semester you say, “My god, will this thing ever end?” You’re trying to get through it and trying to work through this sort of thing, and at the end of it you feel a sense of completion. And that rhythm is so important to understanding the instructional process, not only with technology, but as a faculty member in general. And we have to respect that teaching a long period of time is a difficult task. And you all know that classes are very much different. What goes well one semester does not go well the other semester. We’ve all experienced that, we’ve all experienced that.

Rebecca: I’m living that.

Chuck: Absolutely, and it’s draining, Rebecca. It is draining because you contribute a great deal of emotional intellectual energy to this task. We all do. We were talking about boundary objects. Equality is a beaut, it is a beaut.

Rebecca: You’re talking a lot about some really important resources that are in place at your institution: a pretty expansive support system of technology support, instructional design support, actual technology [LAUGHTER], peer support, time, professional development. That leads me to think that perhaps one of the biggest mistakes people make, or institutions make around blended learning, is not putting enough resources into the system?

Chuck: Yeah. But again, I think you have to be careful with, “If I had more money, I’d do a better job.” The whole notion is not only that, but commitment at the administrative support level, and all particular levels, and basically saying, “We value the teaching enterprise, and we’re going to do everything we can to celebrate that.” That is critically important, because money alone doesn’t do it. You know that as well. I had a friend once who said, “If you had all the money and everything you wanted to do your job, two weeks later you’d come back and say, ‘I don’t feel fulfilled.’” Because it takes much more than that.

Patsy: Well, and it’s easy to hear us talk about all that we have in place now. But understand, we started in 1996, so we didn’t start with all of this. We started with instructional designers with TV trays and laptops, meeting people at the campus coffee shop or in their offices. And it started small and grew as the initiative and the institution grew. So over time, it’s easy to add resources. But I think one of the common mistakes is just underestimating the amount of work it takes to produce a quality online or quality blended course. And not meaning to be a Debbie Downer, but segueing into what we’ve experienced during COVID, we’re very careful to make sure to differentiate what we had to do from an emergency remote instruction, and say, you know, “That is not the quality online learning that we have worked so hard to put into place for the last few decades.” So I think that’s very different, in terms of just trying to survive and keep the doors open, so to speak, for students, with everyone… those who have never been through any training, those who’ve never had any online-teaching experience in the middle of a pandemic with kids at home and scant technology resources… students who don’t have any technology resources, who have major job issues, family issues, in the least-conducive-to-studying room and try to expect everyone to excel. I think that’s really the fact that it does take a lot of work. And so I think that maintaining realistic expectations, as Chuck said, it’s not a one-and-done kind of thing. It’s not a one-and-done when you teach face-to-face either, right? The first time you teach any class, it’s not the best time you teach. You evolve, you learn, you adjust, and continue to try to, hopefully, improve every time you teach. I think that’s true of any of the online or blended modalities as well.

Chuck: Steven Johnson has written a wonderful book called Where Good Ideas Come From. It is just a really, really good book and he has three adages in terms of this. One is… What is the adjacent possible when you start this? What is the reasonable next thing you can do? You can’t do it all, so do the thing that you can do and do it well. That’s the adjacent possible, it comes from a biologist named Stuart. The second thing is a slow hunch. You have to be in it for the long haul. If you look at Darwin, he was in it for the very long haul. And he didn’t even know what he had for several years until he came up with it. You have to commit to the long haul to do it, not a quick fix. None of this is a quick fix. And the next thing is a liquid network. You need to be supported by a network around you to prop you up as you go along. And Rebecca, when you said this in terms of resources, one of the resources is you are in this for the long haul if you’re going to do this. So it takes an institutional commitment, and that always is not there.

Rebecca: What are some of the most promising areas of development in blended and online learning?

Patsy: We’ve had good success using adaptive learning, we have an adaptive learning initiative. We started in 2014, I guess. And again, we learned it takes a lot of work, we use a content-agnostic platform, so Realizeit is our enterprise solution. We do have, our math department uses ALEKS as well. But it takes a lot of work. Faculty want control, and you get control, you have to redesign the course. But you have to actually design all of the content for a lot of it, or at least the majority. And trying to import from another sounds great, but it always still requires work to get it to work well. Though, that’s one of the things that we’ve had good luck with. I think we’re going to see a lot more blended learning post-COVID. We’ve never really promoted synchronous online learning and haven’t seen a big drive to do synchronous until everyone had to do Zoom. And now I think there’s some faculty that are going to run for the hills as soon as they can get out of Zoom mode and never want to look back. But there are some who were already asking, “You’re not going to get rid of Zoom, are you? Because I found it’s really great for me to do these discussions…” So there are people who’ve already, again, figured out instructionally how they can use this effectively, and do it from a distance. And when you’ve got students who might be geographically dispersed or have to worry about traffic, we have parking issues, all of that stuff comes into play. So I think we’re going to see a lot more blended learning in general in the future. And not just in instruction, but also in the workplace. We’ve seen pilot testing in a hybrid work environment now with so many days in the office, so many days remotely. So, I think that’s one thing.

John: If you’d like to learn more about Realizeit and adaptive learning at UCF, we did a podcast with Chuck a while back, episode 30, on adaptive learning, and we’ll include a link to that in the show notes.

Chuck: Oh my God, way back to episode 30. We are dinosaurs, aren’t we?

John: Since a few of our listeners might not have been listening back in the primeval times of episode 30, could you briefly explain why adaptive learning might be useful in supporting students with diverse educational backgrounds?

Chuck: I think you really have to ask yourself, “What is the problem we’re trying to solve in the United States of America?” I’ll get it from a different point of view. I think a problem in the United States is that if you live in a lower economic quartile, your chance of going to and graduating from college is 10%. The odds against you are nine to one. I think blended learning and adaptive learning and some of the things that we’ve learned during this pandemic, are going to help us a long way to begin to solve that kind of a problem. That is, we know that many of our underserved students live in an environment of scarcity. They don’t have enough resources, they don’t have enough time, they don’t have enough transportation, they don’t have enough support from their families. They have simply no opportunity to engage in the university the way we’ve constructed it in the years past. They can’t do it. So we also learned, and very much from several papers and the notion that, if you fix the amount of time that a student has to learn, what they are going to learn is going to become the variable. If what they learn is constant, then the amount of time they spend learning, surprise, is going to be the variable. We’ve confronted this, it’s a difficult challenge for universities. But I think one of the real affordances of what we’ve got is to begin to look at… How can we, as universities, eliminate this horrible educational inequity, which exists in the United States of America? That’s a real affordance that we’re going to learn. John Carroll’s great paper, “A Model of School Learning,” said 60 years ago that we have to find another way, that the students do not start from an even playing ground. And you take an underserved student who comes in and one domino falls, their car breaks, or they can’t get childcare, the entire system collapses on them. They miss one class and they’re behind, they miss two and they’re lost. The optimal choice is to drop out. That is not the choice that we want. And I think the things that we’ve talked about today are going to really help us address these very serious problems in our country. You know it, we are wasting millions of minds in the United States, and we can’t do it anymore. I don’t know if blended learning and adaptive learning will solve it all, but it’s going to go a long way to help, I think.

John: And that issue of income inequality and the growth of that is really troubling. We bring so many students into our campuses, and then we just let them fail out without providing support so that they can be successful. And it’s a waste of resources, and those students are ending up burdened with a lot of debt once they leave, without getting the benefits of that education. And I agree, this is one of the most critical things we need to work at.

Chuck: Let me address the debt problem, too. That’s a huge problem, John. Right now, the accumulated college debt in the United States is $1.7 trillion. That is mind blowing. If that were a GDP, it would be the 13th largest economy in the world. We have got to do something about this in this country, we have got to find a way. And like you just said, tragically, if you drop out, you’re in debt, and you’re in debt for nothing. And Pell Grants are wonderful, but Pell Grants are two semesters. One semester, you get on probation, you have one more to get off probation. If you don’t get off probation, you’re no longer Pell eligible. Do you know how hard it is to bring up a whole semester of Ds in one semester? It can’t be done, can’t be done.

John: One of the things you mentioned is that blended learning is likely to be adopted more as a modality moving forward as colleges have experimented with a wide variety of modalities. But for many campuses, this may be the first time they’re doing it. What are some of the most common mistakes that faculty make when they build that first class? You mentioned before, Chuck, that these classes, the first time you teach them are often unsuccessful. What are some of the things faculty should avoid doing when they build that first blended learning class?

Chuck: I mentioned earlier that when they begin, they view this as two separate modalities and what of my face-to-face… what of the gold standard can I offload? And the thinking of, “I’ll offload the unimportant stuff to the online environment.” It’s wrongheaded thinking. In terms of I think what they have to think about really is, “Are there students in my class who can learn better online? Are there students who function better in the face-to-face environment? And can I somehow encompass that?” And it’s all the things that we wrestle with all of the time in all of the courses. In all of my courses that I’ve taught, I wish I did everything well. Sadly, I do not. I wish I were excellent at everything. And some things, I just cover material. And I think faculty have to come to terms with, “When am I covering materials?” It’s a very introspective kind of thing. And I think you’re not necessarily ready for it the first time because, as Patsy said, I think the first time you’re in survival mode. I think Patsy will agree that the first time you do you say, “What have I got myself into, and why am I doing this?” It is evolutionary. That is a mistake to think about that. And I think a mistake is to think it’s going to go well the first time, it hardly ever goes well the first time. It’s like the first time we ever taught a course. I didn’t know anything the first time I left Wisconsin and came to then-FTU, I didn’t know anything about teaching. So I lectured for three hours. My God, I don’t understand why they didn’t lynch me. And I didn’t know any better!

Patsy: I think Chuck’s right, it’s a lot of work. And so from the faculty perspective, I think trying to have realistic expectations and not look at it as, “Here’s the endpoint of where I want to go, but you have two weeks to do it.” That’s not going to work. Make sure you have enough time. If you have access to resources, I think that’s really important.From an institutional standpoint, we know it takes a lot of work to create an online class. Faculty would say it takes more work to create a good blended class. So you can either have the best of both worlds, but it could also be the worst of both worlds. And you don’t want that to be the outcome. So trying to have realistic expectations and then, from an institutional standpoint, make sure you have the support that you need, not just for faculty, but for students. Make sure that you have the robust infrastructure. And for us, what we found, especially like this last year with COVID, is that we did have a lot of students who were doing those courses, we have a very small number of fully online students that take only online. What our students do is really pepper their courses with a mixture of modalities. So they take an online and a blended and a face-to-face. And we’d like to think that the online and blended help reduce the opportunity cost for getting an education. So they can help them balance their life with education. But we have a lot of labs on campus. They were relying on the computers in the labs, they were relying on the Wi-Fi in their dorms, the laptops that they might have access to, and things like that. So I think just making sure that you have the infrastructure that’s necessary, that you have support to be able to handle any issues, questions, both for students and for faculty. From an institutional standpoint, I think it’s really critical. And if you’re going it alone and you’re a faculty member, and there are people who do that, understandably, if you’re in a situation, start small. Keep your expectations realistic, and grow your course as you learn, as you go along. So keep improving it.

Chuck: One of the strategies… UCF is a very selfish place. In all of these kinds of initiatives, what we do is we cherry-pick the best faculty we can find to maximize the effectiveness of what they’re doing. When we did our adaptive learning things, we had faculty members who looked and said, “Oh yeah, I want to do this.” And we knew they were the best faculty we had and we knew they would make it look good. One colleague in nursing, who has left us by now, she said, “I don’t want any of your tutorials. I just want to mess around with this, RealizeIt, and figure it out for myself.” And she did, and she was marvelous. And that kind of a model really motivates faculty. I would point out so far we have not talked one iota about blended learning. We have talked about blended teaching. That’s what we talk about. We call it blended learning, but we segue, to use Patsy’s word, right away to blended teaching. And that’s an evolutionary thing, Rebecca, that we’re going to have to deal with in the years to come in terms of what is blended learning, aside from the arrangements for teaching. And the other thing that has happened is, you can blend many things. We’ve talked about blending courses. You can blend a university, you can blend all kinds of aspects of this institution in terms of faculty, and students taking online and blended and face-to-face courses. Blending their locations, blending many, many things that people have written about. Our dear departed colleague, Karen Swan wrote about the blended university and several others have talked about that notion of expanding it behind the notion of just a course, that it is much more than that. That is blending a university culture.

Rebecca: Chuck, I’m glad that you brought up students because one of my next questions was going to be: How do we prepare students for this kind of learning? Sometimes they come into an environment. And I know we had a lot of students say this in the emergency teaching that we were doing, like, “I don’t know how to learn online, I don’t know how to do this. I don’t know how to balance my time, I don’t know how to manage different modalities and things.” So what are some things that we can do to support actual student learning in an environment that might be unfamiliar to them.

Patsy: As soon as you said that I gravitated towards the faculty I work with who teach predominantly undergraduates and a lot of freshmen. Now, I don’t want to say they parent them into success as much as they can. But in some sense, that’s what they do. I think it’s probably idealistic for us to assume that freshmen come in and are going to go from being handheld with what we do in K12, to instantaneously going to be able to jump into a course and know what they’re doing and take responsibility and all of that kind of stuff. So faculty are very strategic at using the technology to nudge them, remind them, try to provide some motivation for them to get involved. And I think that’s some of what is important, I think that that helps students ease into this. We find a lot of… maybe it’s because of Florida… but I think Florida requires our high school students to have some online learning experience. But even then, the high school online learning is not the same as what we do. Because again, there’s parent consultations, and there’s a lot more very structured, dedicated time with parents and students that you don’t get, I mean, we just can’t do in higher education. But I do think there’s some strategies. We are seeing a lot of colleges are trying to work to find out who is at risk earlier and hopefully intervene. So what interventions do we have? That really early identification, trying to identify students who are in trouble and making sure they get access to the help that they need early on is important. Some of the analytics work we’re finding, you’ll see, I think, a trend that’s going to continue, as we have more and more access to student engagement, student performance with both platforms and our LMS and all of that. Everybody’s shooting out analytics, right? I mean, we know we track people all the time now in terms of how they’re doing, where they are, and what’s going on. Some of that is helpful, it’s going to be important to sift through the noise to find what’s really helpful for those students who need help.

Chuck: Well, Rebecca, I would say one is you certainly cannot throw these students into the breach. That is the first thing you cannot do, absolutely. In terms of, “You are on your own, and you’re going to do this.” So what you have to do then is really be very introspective in terms of, “What is it I do well in my face-to-face? And how can I translate that into an online environment?” And you know very well, I would rather give a lecture to 5000 hostile politicians than to do a webcast, simply because I am not nearly as effective in that environment. So that’s something that you have to do. And the other thing I think you have to realize is that the student voice has become increasingly important in higher education. We came from a time— at least I did—when you went and you lectured and you left, and it didn’t really matter what students thought. That is no longer the case. They have a voice and they express it continually about the quality of education and they share it. And you must realize that culture is going on all the time. All of those things are coming into play that change the educational environment and blended learning. Clay Shirky’s got a great thing that he did several years ago about… we used to lecture and that was it. But then they wanted to email us and they became really annoying. They wanted access to us more than just our office hours. That was a big change for many of us. And then they began talking to each other about us. And then they began talking to each other more about us. And then they began talking to the world about us. And I’ve used this example all the time, just go on YouTube and pick any topic related to teaching, and a professor, a good professor, bad professor, drunk professor, stoned professor, and you will find videos. Students now share their voice, and they’re part of the voice. And I think one of the big problems that we have to face now is: How are we going to integrate the new, more powerful student voice into the higher education culture? And I don’t think we’ve even begun to address that. And I’ll tell you right now, it is not with student ratings. This is another opportunity, as my administrators say to me, “I have an opportunity for you.”

John: So as we’ve talked about before, last year there was that whole experiment with remote synchronous instruction, where both faculty and students were thrust into new modalities that they were not used to. And there’s a lot of evidence of some significant learning losses, and a change in learning practices that has led to some challenges facing faculty and students as we move back to whatever this new normal happens to be in the middle of a pandemic. How can we address the larger variance in prior learning that students bring into our classes no matter what modality they’re in?

Chuck: Well, what we just experienced is something we knew existed all of the time. What this new experience has done with this very nasty virus is it simply exacerbated those differences. And we are going to have to spend some time, I think, devoting our institutions to how we can recover students. Because clearly, in my mind’s eye, the most vulnerable among our students are the worst impacted by what has happened in the last year and a half. Yet again, John, it is not evenly distributed. Unfortunately, it is unfairly distributed. There’s a great book that I just read, called The Class Ceiling. There’s a distinct wealth advantage in this country, people who are exposed to resources and stuff did much better in this environment than people who are not. And we’re going to have to spend some time trying to recover from that. And frankly, I don’t know how at the moment, I really don’t know how. I hope it hasn’t been too disastrous for us, but it has not been a good scene. As Patsy has said, people have tended to conflate what we did in the last two years with online learning, and they shouldn’t be doing that, but they do it. You know it, you know it, and I know it.

Patsy: We’re also seeing enrollment starting to drop. And I think one of the things that’s going to be really important for institutions to do is keep track of: Who are we losing if our enrollment goes down? Because I suspect it’s going to be some of those people who are our underserved population to really are now, through impact of COVID, forced to do something other than go to higher education. Either they can’t afford to do this, or they have to work full-time more than before, care for family, something… but it’ll be really interesting to see. I don’t think we’re quite there yet. We’re starting to see trends. And I think I’ve heard that across multiple institutions in terms of enrollments. We knew enrollment was going down prior to COVID. But now we’re starting to see some of it is maybe more than we expected. So I agree with you, John, I think it’s going to be important for us to really track that and figure out what’s happening and how we can adjust our instruction and whatever we have to do to bolster and help students succeed, whether it’s due to our instruction, or just no fault of their own due to last year. I think we need to try to figure out how to do that.

Rebecca: There’s a lot of shifting expectations that have happened as a result of the pandemic, in addition to, you know, “What is online learning, right?” And what people have experienced as emergency teaching, but also just other expectations around time commitments, flexibility, taking care of mental health, like all kinds of expectations have just shifted wildly from [LAUGHTER] maybe the way that they were before from both the student perspective and the faculty perspective and the institutional perspective. So I think there’s a lot of shifting that’s going to continue to go on as we all try to adapt to what’s going on around us.

Patsy: Absolutely, I agree with that. I think a lot of human resources departments are going to have to figure out how they need to adjust [LAUGHTER] some of their long-set-in-stone policies because we are evolving. I’m not sure we know where we’re evolving to yet, not there. So we’ll find out.

Chuck: Rebecca, I think you’ve hit on the problem of the century, in terms of… How are we going to re-examine our institution in terms of what our value structure is, what we have been, and what we need to be in the future? I’ve read a wonderful book now called Subtract, where the point is made that every time we try to improve, we add something. We never think about jettisoning anything that we have done, and it may be time for us to begin to consider what are some things that we have been doing that are no longer effective and they need to go. It may be painful, but I I think it’s a lesson we’re going to have to address. It’s just a marvelous notion, you just never think about dumping something. We’re going to add in more… give us more resources, give us more faculty, give us more this, give us more that and we’ll do a better job. Will you? I don’t know, I don’t know.

John: One of the things that I’ve been really impressed with is all the research you’ve been doing on blended learning. And one outcome of some of that research, as well as the work of other people, has been that series of books on Blended Learning: Research Perspectives. Could you tell us a little bit about how this collection of research came about?

Chuck: Well, since I’m the old guy here I can tell you exactly how it came about. Frank Mayadas allowed us to fund three summits at the University of Illinois, Chicago, run by Mary Niemiec, dear colleague of mine. 30 of us got together and discussed this notion of blended learning. And what we did is we discussed, in three groups of ten, three topics: quality—God forbid, I’m glad I wasn’t in that one—and the second one, logistics, and the third one, research. And Tony Picciano and I were in the one: research. And we were in a two-day meeting and finally Tony leaned to me and said, “This is not going anywhere. Where are we going to go with this, in terms of research?” Then the next morning, he got up and said, “We need to do a book.” So the first thing we did is we entered the meeting, he said, “We’re going to do a book,” and we got those people who are interested in doing a book. And then, at that time Sloan was running a conference on blended learning, and so Tony and I went and made a pitch about this initial book that Sloan published in terms of blended learning. And there was tremendous interest in it, and all kinds of people submitted chapters. So we had a book that Sloan published. There was a book and it went out. And then it was successful. In this niche market, a book of 1000 is successful, it’s not like Shades of Grey. And then we got to thinking with Charles Graham, ‘How about doing another one?’ And we went back to Routledge and they said, “Yeah, we see that that book was successful. So we’ll do another one.” And we did the second in the series for them. And then thirdly, they came to ask, they said, “The book did so well…” whatever that meant “…we’d like you to do a third version.” And that’s the motivation for the third book. We got a trilogy on this because Frank Mayadas didn’t like blended learning. But he’s a convert now, he’s come around in doing this. And the last book, we really jacked up the quality because we added Patsy to doing this. And now we’re moving on to a book on analytics and adaptive learning. I’ve got to stop writing books. That’s how it happened. It happened by accident.

Rebecca: Many great things do.

Chuck: Oh, they do.

John: Are there any other things you’d like to add?

Chuck: I’d like to address the fact that I think how valuable your podcasts have been for education and the nation. And I think we’re talking about a classic example of what we’re talking about. Look what’s happened over the last years: the value and importance of doing podcasts. Obviously, yours is one of the premier, and I’m not stroking you, you know I don’t need to stroke you. But the notion of this is another modality for learning that we never really considered, right? And this is what we have to do. What are these things? Cherry pick the things that are really working.

Rebecca: So we always ask and wrap up by asking, “What’s next?”

Chuck: Patsy, what’s next for you? Cause I know what’s next for me.

Patsy: Well, yeah, what’s next… wow… is surviving COVID. That’s number one. I think that’s our top priority right now, making it out of the pandemic and figuring out where we’re going to evolve to, as we’ve already talked about. Chuck and I have this book that we’re just now working on, which is data analytics and adaptive learning. So that’s coming down the pipe. And I don’t know, for me, it’s like trying to figure out how to continue to help facilitate research and enjoy what I’m doing, trying to make a difference with as many students and as many faculty as we can. So yeah. That and counting down the days to retirement because I know that’s what Chuck’s going to say. But I have a lot longer, I think, on the track than you Chuck, right?

Chuck: This is my 52nd year at UCF, formerly FTU. I think what’s going to be in the future, for me, will be retirement, but it will also be trying to reflect on what we have learned in this educational enterprise and what we need to learn as we go forward with this. Personally, it is going to be, John, and you know about this, I will be moving over to working with our underserved communities to try to solve the problems of inequity that is so crippling our educational system, and we need to work on that. But those are the kinds of things. I guess I would say thank you for having us again, and we look forward to hearing what we had to say. Right, patty?

Patsy: Yeah, it’s been a lot of fun.

Rebecca: Thank you both.

Chuck: You guys are great.

John: One thing I do want to mention is that we also had a podcast with Chuck, where he did talk about some of the ways of working with this in Episode 115, where Chuck and Harris Rosen talked about the Tangelo Park project.

Chuck: Just as an aside, the Travel and Leisure Co. has adopted another community based on what we’ve done, John, in the Eatonville community. And we’re very close to a foundation in the Midwest doing the same thing. So we are making progress. Hard as we try to get people to replicate it. I think we’re at four now.

Rebecca: That’s great!

Chuck: Yeah, it is great. It is great. And it really does work.

John: Well, thank you. It’s great talking to you always. And we’re looking forward to sharing this with our listeners.

Chuck: We really appreciate your thinking of us. We really do. Go Lakers!

John: Go Knights!

Chuck: Take care, everybody.

John: Thank you.

Chuck: Thank you so much. Bye bye.

Patsy: Thanks, guys.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

217. Grading Justice

Traditional grading systems can encourage students to focus on their grades rather than on their learning, and favor continuing generation students who are more familiar with the hidden curriculum of higher ed. In this episode, Kristen Blinne joins us to discuss grading strategies that promote equity and encourage learning.

Kristen is an Associate Professor and Chair of the Communications and Media Department at the State University of New York at Oneonta. Kristen is also the editor of Grading Justice: Teacher Activist Approaches to Assessment. Judie Littlejohn, the Instructional Designer at Genesee Community College and a frequent guest on the podcast, joins us again as a guest host.

Shownotes

Transcript

John: Traditional grading systems can encourage students to focus on their grades rather than on their learning, and favor continuing generation students who are more familiar with the hidden curriculum of higher ed. In this episode, we discuss grading strategies that promote equity and encourage learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Kristen Blinne. Kristen is an Associate Professor and Chair of the Communication and Media Department at the State University of New York at Oneonta. Kristen is also the editor of Grading Justice: Teacher Activist Approaches to Assessment. Judie Littlejohn, the Instructional Designer at Genesee Community College and a frequent guest on the podcast, is joining us again as a guest host. Welcome, Kristen.

Kristen: Good morning. I’m very happy to be here with both of you.

Judie: Good morning.

John: Today’s teas are… Are you drinking tea?

Kristen: I am, it is a jasmine green tea, my favorite of the tea world.

Judie: And mine is a Twinings Lady Grey.

John: And I am drinking Tea Forté black currant tea, which is one of my favorites, with some honey from Saratoga Tea & Honey. I think you both met through Judie’s work on the FACT2 Subcommittee on Social Justice Assessment. Could you tell us a little bit about that?

Judie: The subcommittee is part of the larger Innovations in Assessment Committee and Chilton Reynolds is one of the co-chairs of that, and he is at Oneonta where Kristen is. And so he notified our group that she had published this book. And Chris Price in our group reached out to Kristen and asked if she’d be willing to meet with us. And she did, and we had a great conversation. Unfortunately for me, I was driving that day. So I was on the thruway trying to participate as well as I could. And it was great, it was a very engaging conversation, a nd it was great to hear from Kristen and hear her enthusiasm and all her great ideas. So it’s great for me to meet with you again and to finally see you this time. [LAUGHTER] So welcome, I’m glad to see you here.

Kristen: Thank you, I was so excited to receive that message about the FACT2 Social Justice Assessment group. I didn’t even realize that that existed at the time that that email was received. And so I was truly overjoyed that these conversations were happening in a coordinated way in the system, and to be able to just have that moment to jump into the conversation here, kind of what the group is doing and the vision for moving forward. And it’s exciting to see even the website materials that are up that are situating these conversations for a broader audience.

John: We’ll share a link to the website for that group in the show notes. So, we’ve invited you here today to talk about this book. Could you tell us a little bit about how this project came about?

Kristen: Yeah, so the Grading Justice book project really grew out of a few different streams in my experience just teaching, but also my experience as a learner, as a student going through a system. I have to say that it was born out of a lot of frustration that I had as an instructor. [LAUGHTER] And I hate to frame it in that way, because it’s kind of a negative framing. But I think most instructors can agree that we spend a lot of time grading, and the grading comes with a bunch of challenges, especially if you’re doing any kind of activist or social justice work or you’re trying to create equitable learning spaces. Trying to do grading in a fair way that best meets the goals of the course and to have students on board with that process is difficult work. And so in my case, I looked back to my experience as a student when I tried to build my classes coming in as a college instructor. And I realized that as a student, I really didn’t have a very positive relationship with grades. They didn’t matter to me that much, I was always in it for the learning, but I had a lot of difficulty going through the system because of the emphasis on grading. I wanted to be a learner and I wanted to explore my creativity in ways that maybe didn’t fit with the limited grading structures that I encountered as I went through K through 12 and into my college experience. It wasn’t until I finally landed at Goddard College in Plainfield, Vermont, an institution that doesn’t do grading, that I’ve really embraced my learning in a new way. So when I came into my own teaching process, I realized it’s like, “Wow, okay, I have to now impose these grades on the learners in my classes. And how can I do that in a way that honors the journey I went on, recognizing that other students may be also experiencing the same struggles that I had?” So I came to this Grading Justice project because I wanted to have these conversations with educators, really across disciplines, but the book is focusing a lot on people who are working within the field of communication and critical pedagogy but also critical communication pedagogy. But my hope is that it has appeal to a wider audience than that. Aside from the frustrations, another thing that I noticed because I do a lot of social justice work is… How do you assess social justice work in an equitable way? So that was one of the questions I came into the project with: How do we assess social justice work? But then… What would a social justice approach to assessment look like in our teaching and learning processes, especially in the realm of critical pedagogy, because it was part of a critical pedagogy book series? I’ve noticed that a lot of people who embrace critical pedagogy, they may still be using traditional learning systems. And so how do you, as an educator, work with that tension that might be coming with reinforcing systems that may be perpetuating inequality while you’re also trying to undo systems of oppression and engage in power sharing with students? And so I was finding this contradiction that inspired me to want to really pursue this in a very serious way.

Judie: A major theme of the book is that grading systems can either perpetuate inequality or work toward equity and justice. How might traditional grading and assessment systems perpetuate inequality?

Kristen: So one of the things that really spoke to me as someone who was trying to navigate how to create a more equitable classroom, because I’m really invested in making my classroom as diverse and inclusive and accessible as possible, is having conversations with students about assessment. Because what I found over time is that students, as I mentioned about myself, many of them have a pretty negative relationship with grades and grading. And a lot of them don’t see them as accurately measuring their learning, at least in the experiences I’ve had in having these conversations across my classes. I had this lightbulb moment when we had a guest speaker come to SUNY Oneonta, Ernest Morrell came as part of one of our teaching institutes. And he said something in a small group session that, really, I think about a lot in regard to this question about grades perpetuating inequality. He said, “We need to stop measuring students’ success against failure.” And in that moment, when those words kind of tumbled out of his mouth, I just was like, [EXPLOSION SOUND]. It occurred to me, I always have wanted to encourage… I don’t want to say encourage failure in the class… but I want to encourage experimentation… classroom as living laboratory where people can try things and have it maybe not work out. An example I could give of that is I taught public speaking for many, many years. It’s a class that students have a lot of anxiety about taking in many cases, and there’s a struggle there. And so one of the things one of my students did once is they decided to sing a poem about Rachael Ray. I wasn’t sure if this was a great idea, but they did it. And if you ask them, they would say it was an utter failure. And it didn’t work out in the way they wanted it to. But why would I grade someone on that failure? And say, “Well, it didn’t work out as you wanted to, the audience wasn’t really that invested in that approach of singing a poem.” There’s a way that that could actually have worked very negatively and inspired a student not to take a chance in this case, but it allowed them the space to try something. So one of the things that I think about with this quote of Ernest Morrell is stop measuring success against failure. What does that mean if we think about grades in regards to equity and inequity? Well, first off, what does grading do? It quantifies our learning. It creates some kind of measure that we’re using as a tool of comparison. It operates, in my opinion, a lot like a credit score. It gives us a numeric value or worth that can grant us access to or limit us from different opportunities. Those opportunities can be admissions into something, whether it be grad school or undergraduate, teaching assistantships, it can offer us opportunities for funding and scholarships. There’s lots of ways that it serves as this gatekeeper process. It’s also something that labels us as learners: the A-student versus the C-student versus the student that we might say is the failing student. And in that way, it can stigmatize pretty extensively. It’s also, in my opinion, a system that both rewards and punishes. And what it’s rewarding and punishing is really dependent on the way that the instructor situates grading in their course, of course. Beyond that, I would argue that, as many other educators working with non-traditional assessment might suggest, that it oversimplifies complex learning processes. It creates a snapshot of a moment that doesn’t give us the full context of what’s happening in that student’s life in that moment. So we don’t have any idea most of the time about all the many struggles that our students are facing, until often maybe it comes about at the last part of the semester, when they’re trying to finally disclose some of the struggles that they’ve had. Aside from that, of course, beyond that comparison, and then like a mechanism of standardization, it also is something that’s applied pretty inconsistently across instructors, which in some ways renders it as a kind of arbitrary measure in the sense that my B isn’t your B. They don’t mean the same thing necessarily. So how are students supposed to make sense of a process where they can’t recognize this standard across this thing that’s supposed to be standardized? Because it invites, I think, when you take all those factors into account together, an opportunity for students to look at this system as a kind of game that they have to navigate. And not a game in a fun way where we’re enhancing our learning, but a game in the sense that, “All I need to do to be successful in this system is to learn what the professor wants. How do I make sure that I meet this deadline or I can amass as many points?” And it becomes this process that for many, I think, is very decoupled from the learning itself. And then we can, of course, see how this snowballs into all kinds of conversations about grade inflation and its relationship to evaluations of instruction and marginalization in regards to the bias that can happen as instructors grade across the different learners in their classes and their abilities and understandings. And all of that is, in the end, we need to start asking that question, I think, more and more: “What do grades mean? What are we actually trying to measure to kind of undo some of the ways in which grades function in an inequitable manner?”

John: And so students spend a lot of their time focusing on learning the rules of the game. But it’s a different game in each and every class that they take, which I think you’re arguing would distract them from actually focusing on learning the material that we’re hoping they get out of the course. Your book is an edited work, where you have a number of contributors. How did you solicit the contributors for this project?

Kristen: I first started by doing an open call in the National Communication Association listserv. At the time, it was called CRITnet, which is Communication Research in Theory Network, it is now called something else. But at the time, I sent out an open call there. I also handed out flyers at the national convention we had in Salt Lake City the year that I was putting together this project. And so I was really excited by the people who responded to that call. I also reached out to a few of the contributors based on work that I knew that they were doing in this area to try to round out the collection. And I can imagine that this could have had multiple volumes, just with the really interesting work people are doing in grading and assessment. So the chapters in the book are diverse in their scope, and even still, they paint a very small picture of a very big conversation. So we had a chapter that focused on grade inflation, just about the rhetoric of grading. Chapters that looked at team teaching and types of collaborative course construction. Assignments that are focused on, I say the “borderlands” or looking even just at activist work in general. And then chapters that explored going into critical Universal Design for Learning, moving into discussions about teacher evaluations, but also just assessment more broadly. And then my own chapters focused on my experiments with non-traditional assessment, the experiments I’ve had in that realm, and the work that I’ve done in addition to setting the stage through the introduction. And I’m so grateful for the collaborators that were part of this project. I mean, I really see it as our book. Of course, my name as the Editor, but we were a team. And we were a team that went about this project in a way that I thought was really beautiful, because I did invite the chapter collaborators to read each other’s chapters and offer feedback as part of the process in addition to my feedback. And we also then worked together at a national convention post the publication of the book to do a short course where we actually taught about our respective chapters. And it was well-attended, and we had some really robust and interesting conversations about how instructors could carry these ideas forward.

Judie: Kristen, in chapter seven you discuss your own experiences with non-traditional assessment. Could you tell us a bit about how your assessment strategies have evolved?

Kristen: I’d love to explore that more. As I said, I feel like my journey with learning to do assessment really started with my work being a student at Goddard College. I was building my own course plans as part of the way that Goddard is set up because it’s working in a tutorial model where, in my case, at the time I was there, you did 15 credits with one faculty member. You built the content for that course and they helped guide you through that process. At the end of that journey, you did a self-evaluation, and they did a narrative evaluation that went into your transcript. So that really, for me, set the foundation for this experience. But then when I went into graduate school, I had the opportunity to work with one of the faculty at the University of South Florida, Mariaelena Bartesaghi, who was, at the time, working with our Interpersonal Communication course. Many TAs oversaw that course. And she had done a grant that she had designed to reimagine that course around process pedagogy and portfolio work, and was really drawing on the work of Peter Elbow and Jan Danielewicz’s work on the unilateral grading contract. And so that was where I really started dipping my toes into finding a way outside of more traditional grading systems when I was a teacher versus a learner. And so I have to say that my own approach started with that approach, the unilateral contract. And for those people that may be unfamiliar with that, it’s a process that really creates the B as a baseline, the B grade. So it’s behavioral in the sense that students are assigned a series of expectations that they must meet in the semester and if they meet those expectations—whether it be number of classes missed, or work turned in, following the instruction, so it’s up to the instructor to determine what those criteria are—then they’re guaranteed that B grade. And anything that goes above a B is up to the discretion of the instructor as far as whether they’re working with a plus/minus, a B+, A-, A. And often in that case, there may be a focus on the quality of the work, there may not be, depending on the instructor and how they implement that plan. One of the things that I found in Elbow and Danielewicz’s system, is that they maintain a strong hold on the course policies. But I was more interested in trying to find a way to go more in the direction of your assured work and thinking about how I could share power with students and have them collaboratively construct course policies. So pretty quickly, and what I kind of call in the text, the guaranteed B approach. One of the ways that I started was always about collaborative course content construction with the students. So consensus process to build ideas about: What does participation mean? How does it function in our class? What role should attendance play in this process? What would an A look like based on the assignments that we’ve done this semester? And so letting them have a voice in that and having a lot of conversations in the context of the class about grading and assessment. So really involving their voice in the process was important to me. I will say that, just kind of broadly, as far as my own approach, I don’t do tests and quizzes, I prioritize other types of assignments. I don’t use percentages or points. Early in my process, I used a lot more markers in my grading. So like a check, check-minus, check-plus, or I would use markers like “meets criteria for a B,” “does not meet criteria for a B,” “exceeds criteria.” Because at the time, I still thought that students really wanted that marker to help keep them on track. And that part has really evolved for me. And I think earlier in my process, I was a lot more attached to attendance and participation models. Even though they might have been student-identified and selected, I put more weight on it than I maybe do at this point. So from the guaranteed B approach, my focus went further into a tiered method that I call kind of a pick-a-plan grading where it was, I think, most similar to what some may use, like a labor-based grading, where you’re actually doing more work for higher grades. And I found that students really responded very positively to that. It still had that kind of guaranteed B if you did this level of work based on the same kind of criteria, but then it allowed more work equals a higher grade. You weren’t guaranteed that more work would equal a higher grade, but it allowed them to make choices upfront saying, “Well this semester, I have a lot going on, so I’m going to choose to do less work,” and that’s okay. And I had quite a lot of students do that. From there, my process evolved to I think what I consider one of my favorite approaches in my experiments over the years was what I called the 100% participation or engagement plan, if you want to use it as a plan, which I think scares a lot of instructors, that idea [LAUGHTER]. Because participation is such a murky realm already for many. What does that even mean? What does that look like in practice? So for me, what that meant in my class is that the students dictated: What does 100% participation look like? And as part of that, I required that students meet with me to discuss their own participation and engagement in the course, we create a consensus around that. And then at the end of the semester, would actually meet to finalize each person’s grade based on what the class determined as their overarching grade categories and the students’ own assessment of their participation. In the text I also talk about the group-focused version of that, which I won’t get into at this space. But the last thing that I talked about as part of my experiment, is that in some semesters, I would just ask the students to pick what kind of assessment they wanted in the class. So I’ve given them some of these ideas: “Do you think you want to do a guaranteed B approach? Do you want to do more of a tiered pick-a-plan approach? Do you want to do an approach where you are doing 100% participation and engagement?” And so it would not be uncommon for me to have some semesters where every class had a different assessment system, based on what the group themselves decided. And key to that is that I always just remained flexible and adaptable to shift it if it wasn’t working, because we had other tools that we could draw from. Another way that the select your own assessment process has worked for me is that you can ask students to decide what they want to be assessed on in the class, this paper versus this—I don’t do tests, but you could do it with tests—as a method. So all of those were really great experiments for me, and “experiment” I almost think it sounds negative in the way I’m saying it because I tried and I had a lot of failures trying these different systems, a lot of struggles, and things that came up. And it occurred to me along the way that maybe some of these practices, while they were kind of masquerading as being more just and equitable, I was maybe falling into some of the exact same traps that I would have been had I been using points and percentages in a more traditional approach. And so that was another huge “aha” moment for me that contributed to the construction of this project. How do I actually embody a system that maybe isn’t falling into some of those traps, even if I think that I’m doing that work? And that’s where I found myself developing my approach to ungrading that I call “awareness pedagogy.”

Judie: That’s interesting how you’re trying to do the right thing, you’re trying to make something better, and then you find out or realize that maybe it’s not better. It’s frustrating. I’ve run through that with offering extra credit, and then I read an article that said that extra credit inherently favors the students who are already doing the best and have the best time and the best preparation. And I thought about that for a long time and realized, “Yeah, I’m doing a disservice to a lot of students by adding to the pressure with more extra credit.” And it wasn’t easy to get to that. I kind of had to see it and then reflect on it for a while, and it was frustrating. So, I don’t do that anymore.

John: Especially when students are asking for extra credit, especially late in the semester. Explaining to them why you don’t do it, though, perhaps could be a useful learning experience for them too.

Kristen: It’s interesting. Because I can’t even remember the last time a student asked me about extra credit in a class. [LAUGHTER] Maybe they just assumed that it just doesn’t exist.

John: Both faculty and students generally find grading to be a very unpleasant experience. And I think many faculty would like to move away from this to some extent. But they may be facing pressures, especially if they’re untenured, and probably especially in the STEM fields, to use traditional grading systems or grading approaches. Are there some strategies that faculty could use that they want to move away from really bad practices to somewhat more equitable practices of assessment?

Kristen: Yeah that’s a great question. And absolutely, I think it’s so important to just acknowledge that not everyone has the same access to actually utilizing a non-traditional assessment in their classroom, whether it be because they’re mandated to assess in a specific way, “Here’s the syllabus, here’s what you’re teaching, go forth and meet your class.” Or if it’s because they maybe are in a marginalized space in the process or they’re in a precarious position. So some of the things that I think that I would invite instructors to consider to lessen the impact of grades and to maybe make the grading process more purposeful is to just, first and foremost, revisit your course policies. And whether you collaboratively construct them with students or not, revisiting our course policies is a really interesting way that we can start to look at the consequences of what we’re setting forward for students to do in our class. So as I said, I used to lean a lot more heavily on the role of late work in my class, like the no late work. And also just having more strict attendance policies because, again, I was thinking about it in terms of this behavioral approach that Elbow and Danielewicz had outlined. And I just stayed with that for a while, and it made sense to me at the time. But after I started seeing that maybe reinforcing these non-academic behaviors was actually not in accord with the learning goals that I have for the class. And I wasn’t really taking into account the whole picture that the student was experiencing, I started to go back and go, “Okay, how can I soften the role that those behavioral policies play in my course?” I think that’s one thing that instructors can do: look at what they’re determining in regard to attendance, late or makeup work, participation in general, and just having conversations with our classes about what it means to participate in the context of this class. What does attendance do or not do in regard to your capacity for success in this course? And also, just more broadly, makeup work and late work is something where we can see a lot of students really suffering in their grades in the end, and it could be that they have any number of things going on that they’re not ready to disclose to us. So that’s one thing. Another thing is aside from course policies in general, and rethinking the non-academic things that we’re actually grading is to consider the possibility of doing minimum grading. As outlined by Thomas Guskey or Douglas Reeves, this idea that eliminating the zero in our gradebook. The zero is a powerful tool that can really keep students from progressing. They may have had something come up that caused them to miss that assignment, maybe were not willing to budge about it, but that zero is going to have a ripple effect that it’s very difficult to recover, depending on the weight or the points of that assignment. So consider minimum grading, a 50 If you’re working with 100 points. What if you give them a 50 versus a zero, that does something that can maybe allow the student some possibility of bouncing back. Because how does it motivate someone if they have a zero and they know they can’t recover? It’s like you’ve lost them. And then what is our learning doing in the class? Are we just going to let them be adrift and not try to find a way that we can move forward? So I would encourage that. I would also say that if we’re thinking about it from a social justice standpoint, that we should stop averaging when possible… again, because if we look at grading as a kind of reward or punishment, averaging can take that moment where we did that zero or we froze them in that time when… Who knows what happened? Maybe they lost a loved one, maybe they just received a very scary diagnosis of their own health, maybe they’ve been working more than 40 hours a week or something that we just don’t know about… they’re experiencing trauma at home or in a relationship. And so averaging, I think, is one of the things that I would encourage instructors to consider reducing their reliance on or even curving. But that, of course, gets us into a whole ‘nother realm of like, “Okay, well, how does that impact now, if you’re going up for rehire, or on the job market, or a tenure and promotion? If you start putting these policies in play, how will your colleagues understand them as they’re assessing you?” Which, in itself, requires a lot of additional labor which I think keeps some instructors from stepping their toes into these different possibilities.

Judie: I read in Jesse Stommel’s blog post, “How to Ungrade,” where they start off their class with maybe the first few weeks the students’ work is not graded, it doesn’t have a point value, it just is for them to learn how to do the assignments and get the feedback. And then they could start from there for some sort of point value, so that they have a chance to grow accustomed to how the course works and what the expectation is before they’re graded.

Kristen: And that’s a beautiful way to think about it too. And that was one of the things that I’m glad you raised it, because I also wanted to mention that, is building in more space for pass/fail opportunities or for ungraded assignments. And even extending that further is maybe building in more space for peer evaluation and self evaluation. Having those conversations with students about what constitutes good peer evaluation or peer feedback and creating guides for that. And even having conversations with your class about what constitutes a good discussion in class, if you’re a discussion-based class, because maybe not a lot of students have actually learned how to engage in productive discussions or dialogues, especially across difficult topics. So creating those opportunities to have space in your calendar to allow those conversations to happen, I think is really, really helpful.

John: You mentioned how students can learn from their mistakes. And as academics, we know that we often learn the most by trying something and failing, and we want to encourage students to do that. But when we use high-stakes exams, that certainly deters students from taking risks and trying new things. What can we do to help relieve some of that pressure to encourage students to be willing to learn from mistakes, because that’s not something they’ve learned from their past educational experiences?

Kristen: Absolutely. And I think that’s why I prioritize revision in my classes, like every course I teach has some element of revision built into the process. So for example, in the courses I’m working with this semester, everything is ungraded up until the final project, which is the primary way in which the course grade is determined. So everything’s a draft until that final project that they turn in. So they have their self-evaluation process as part of that. They have a peer feedback component, and then they get my feedback, they build in that revision. And then we actually do our grade consensus process at the end when they’ve gone through all of that revision and feedback, so that they have that last layer of opportunity to revise it even further towards their grade goal in the semester. And so I think revision is one way that we can do that. And not every class lends itself equally to utilizing a revision project, I realize that, but there are ways that you can do it in a manner that I think students can still gain something from, even if it’s just to show them that not every assignment is one and done, that it’s a process that they can find ways to improve it and to gain new information.

Judie: If we look at the students that are in a classroom, how can faculty leverage the diversity of student backgrounds to create an equitable learning environment?

Kristen: So one of the things that I really wanted to share with you all that I tried this semester for the first time, and I thought the results were pretty great. At the beginning of this semester, I asked my courses to do a syllabi inventory of their classes. And we all know, I know we hear instructors say it all the time, like, “It’s on the syllabus. Return to the syllabus, it’s there.” And it’s this often unread document that creates the roadmap for everything that’s ahead. I know that there’s lots of ways that instructors try and get students to read the syllabus. They create syllabi quizzes, and they do all these little things to get them involved. But I thought, ‘Well, why not ask them to go into their syllabi for all of their courses this semester and to answer some questions?’ So I just used Microsoft forms to build something for them where they went in and they told me about the number of courses they were taking. They did a comparison of the attendance policies across their courses, how participation was defined across their courses. They looked at late work policies, grade grievance policies, policies around accommodation and support, policies that may be focused on communication in the classroom, or specific instructions about how to communicate with your instructor. And not just by email, but how to address them or other things that instructors mention. There were also questions about behavioral focused policies. So what are the things that might cause them to be penalized in a class, whether it be disruptive use of cell phone or technology in the classroom, whatever it is. At the end of that process of looking at the similarities and differences across all their course syllabi, to tell me what their ideal course would be if they were building it based on what they saw in the classes that they’re taking in this semester. And then I use that data in the next class session to say, “Well, how can we build this class to take that information that you’ve gained and to create policies that would be compassionate, but also hold you accountable for your learning choices so that you’re getting the most out of this class that you can?” And it just was such a fun conversation. And I got a lot of feedback from students that was unsolicited in the sense that they said, “Well, I just had never looked at my syllabi in this way before. And I actually feel a lot more prepared for the semester now that I actually compared.” And it gave me a sense of just, like, how much stress students face trying to navigate the different instructor expectations. I was, I don’t want to say shocked, because we’ve been doing this for a long time and you have a sense because you talk to your colleagues about what they’re doing, but just the level of work, expectations that were there, and the huge spectrum from very flexible to very inflexible, and how it would be a full-time job for students to just navigate those expectations. So it makes sense to me even more now that we’re maybe putting our emphasis in areas that we could rethink, as educators, to help students get the most out of their learning, and less about having to make sense of what we want in a class.

Judie: I can relate to that, because I teach history online, and I keep weekly schedules. But if students need more time, they just have more time, they have until a date at the end of the semester when everything is due. And I try to re-emphasize that you take the time you need, it’s fine, there’s no “late,” there’s no penalty, just relax. And when you can do it, you do it. But then I send them reminders that this phase is ending, this next one is starting. So it’s a good idea to try to stay on track. And often I’ll get emails from students saying, “I’m so sorry, this is late. I understand there might be penalties.” And I think, “Why do they understand there’ll be penalties?” And all sorts of apologies. But then I kind of took a step back and thought, “Oh my gosh, if they’re juggling five and six classes and all these different policies, of course they’re confused.” And I just try to write back and reassure them that I understand that people have different situations, and you have to take the time you need without penalty. And please, don’t let my dates add to your stress. But it’s got to be really difficult for students to try to keep track of everybody’s policy on top of all the reading and work that they have to do in all their courses.

Kristen: Absolutely, yeah. It was a real eye-opener for me to just see the data in front of me. And to contextualize that with the broader conversations we had about just their general relationship with grades and grading and their own perceptions of whether grades accurately reflected their learning in their classes. So it invited us into a space that I thought was vulnerable, but also really powerful for imagining a way to do it differently. And of course, we have this backdrop that we’re facing with the pandemic and how campuses are navigating the return from remote learning to in-person instruction and the stresses that come with that as students maybe are now navigating not only different policies, but different platforms. So that was another question that I asked is, “How many classes are you doing remotely versus in person? And how is that impacting knowing what you’re doing and when and where and how in your process?” Again, a lot of stress. As a new Chair, I can say that I have had so many conversations with students this semester, just in tears, trying to make sense of maybe unclear expectations that are being set forward in their courses, or just lack of communication that’s happening. And I just get this sense that so many of them feel adrift. And I know that, at least among my colleagues on campus, our motivation has been challenged because you go out and into your classes and you maybe see that people aren’t as engaged or connected as maybe previously pre-pandemic. And it’s like you feel like you’re tap dancing really vigorously to get everyone to be part of a process, and it’s this delicate dance we’re all doing to make this matter.

John: You started this project before the pandemic but it was completed during the pandemic. How did the pandemic influence the final work on the book? And do you think the experience that faculty had in more directly observing some of the challenges our students faced might make them more open to considering non-traditional grading practices?

Kristen: I definitely think it has made faculty more open. I’m part of a lot of social media pedagogy-focused groups where there’s been pretty strong debates about what we’re doing in this moment as we teach and learn in a pandemic. And some people feel pretty strongly about maintaining this perception of rigor and these strong standards as a way to keep everyone on track and hold on to that perceived norm that we had pre-pandemic. And then there’s others that have, I think, done so much emotional labor, bending over backwards to be as compassionate as possible to recognize just the weight that everyone’s carrying in regard to just the heaviness of this pandemic, and the impact it’s had on us personally, professionally, and just socially. And so, especially at the earlier stages as this book was coming out, I wanted to go back in before it actually went to print to talk about the ways in which institutions had transitioned to different grading models in 2020, to try and attend to the impact the pandemic was having. So we learned that, institutionally, while it seems like you can’t decouple traditional grading systems from academia in general. We did. We went into so many institutions, created pass/fail options, credit/no-credit options, a variety of different system-based changes where students could not have their GPA directly impacted by the pandemic. And then, of course, we saw that happen, and then we went right back to the previous methods and models pretty quickly after that semester, returning to this norm. So I say “norm” kind of in air quotes, but it reminded me that we can do it, we can make some transformative changes in our learning if we want to collectively embrace that. But it’s also something that I think people still have a lot of discomfort about. I think many instructors, at least I hope, want their students to succeed, and they want to be compassionate and to help them succeed. But we don’t always know the best way to do that because we are managing, ourselves, a lot of expectations just in our own responsibilities and roles. And we’re also tired, and many people are stressed, and just definitely surviving, not thriving, in this moment. So I know that people are also getting fatigued… compassion fatigue happening. They’re becoming a little bit less trusting of the many emails that students are sending, asking for exemptions and extensions and extra credit. And so I think we’re in this moment where we’re all invited to say, “Where do we want to go from here? What kind of learning model will best meet the needs of our future generations because so much is impacting it?” We have this huge political opposition that’s permeating our social world and conversations in the public sphere. We have this fear and anxiety about climate change. We have just so many things going on that this is a beautiful moment for us to imagine a new way forward that could best meet everyone’s needs, I hope, to thrive more in our learning environments.

John: Behavioral economists have found a lot of evidence of status quo bias, that people tend to do the same things in the same way, unless there’s some sort of disruption. And I think this pandemic, and all the other things you mentioned, have led to a disruption which makes possible transformative change in ways that would be much less likely to occur at the same rate in other time periods. I’m hoping, at least.

Kristen: Exactly, and I know I think about it a lot in just regards to changing our own communication patterns in our relationships. I mean, one of the ways that we go about doing that is to do what comes unnaturally, to do the opposite… cultivate the opposite, jostle ourselves out of our norm so that we can imagine another possibility. So I’m hopeful. I think that we’re seeing just these conversations just taking hold in a lot of ways. I mean, I lean a lot on the Facebook group, Teachers Throwing Out Grades. It’s a big group of people, about 12,000 people at this point, that are having these conversations just in that one space. I mean, I know they’re happening in all kinds of other spaces. But that one is one that I like to follow very closely. And even, if we’re using Facebook as an example, the Pandemic Pedagogy group is where you see a lot of people having debates about these issues that we’re facing as teachers and learners.

John: Yeah, those groups have been really helpful in the last year and a half or so, as well as the Twitter conversations.

Kristen: Oh, absolutely. Yes, the academic Twitter and other spaces.

John: And we’re recording this at the end of the semester. And you mentioned all of the emails and requests from students. And one thing I’ve tried to convey to my students is, unless there’s really extraordinary circumstances, I’m not going to make special exceptions only for the students that approach me, that I’d rather build it into the course structure itself so that those opportunities are available for everyone. Because, otherwise, the students who are most likely to request extra credit and so forth are the students who generally come from continuing generation families. And there’s a lot of students who don’t realize that they have that opportunity to request things. So in general I try, in my courses, increasingly in the last few years, to build in more opportunities for revision, for submitting things late, and so forth. But those are open to everyone on an equal footing. And I say if they’re having some major crisis, then I’m happy to talk about it. But in general, I think we have to be careful not to only make exemptions for those students who come forward. It’s much better, I think, to build those opportunities for everyone, including those who might be afraid to ask for those special cases.

Kristen: And I agree, and I definitely try to structure my courses in that way. But I’m thinking now from kind of the perspective as a Chair or just as a Faculty Advisor… What do we do with the students that, in those classes where they have those opportunities, they’re still succeeding because those opportunities exist, but they also, maybe in three of their five classes, are dealing with these very rigid policies that maybe instructors are not understanding that they just got a major medical diagnosis, or they’re on the verge of needing to take a medical withdrawal because they’re having a mental health crisis? And then how do we help those students still succeed? And those are questions that I don’t have an answer for. I keep asking myself that because I know that, when I think about grading broadly, often grading is a disruptive tool that impacts the relationship between the teacher and the student. And so maybe students don’t always feel comfortable coming and talking about it until the end where they’re saying, “I didn’t get the grade I was seeking,” perhaps, or, “Is there anything I can do to recover the grade?” But then where can we plug in in other spaces, I think, instead of being in that instructor role, but in advisor roles or in other, like, role-model positions with students where we can help them? Those are questions I keep asking.

John: Certainly there’s a difference in instructor flexibility, which is a major problem that students face. And as we talked about before, they have very different requirements in each of their classes. And just yesterday, I had five requests for extra credit. And each time I referred them to the opportunities that were already built in. And I said, “I’m not going to ask you to do extra work when you haven’t done some of the required work that you still can do. Before you ask to do something more, maybe you should look at the things that are available for you that you’ve been asked to do since the start of the semester and start there.” And that doesn’t always get the most positive response.

Kristen: And that’s where I think self-assessment can be a really great tool that, if instructors actually build in, whether it be on an assignment basis, or just the broader course, a self-assessment. I know that I’ve worked with asking students to kind of keep a log of their process in the class, the work that they’re doing in-person or out of class. If you’re remote, of course, those distinctions aren’t important. But it’s been helpful, I think, for us to have those honest conversations at the end of the semester. It’s like when we’re talking and you’re saying, “Well, I think I deserve an A in this class.” But then I’m saying, “Well, but you weren’t there for more than 50% of the semester. And these are the assignments that were not turned in. Please help me understand your perspective so that I can say, ‘How is that fair for those students that maybe have been there and participating in a way that you weren’t. Help me understand how you’re understanding this.’” [LAUGHTER] And then I would say 9 out of 10 times that student comes back and says, “Wow, that probably isn’t fair to the other people.” So anytime that I think we can pull back the curtain and just have these process conversations, I just continue to be so inspired by what can come out of them. Of course, you’re always going to have the student that’s like, “I deserved an A. I know I didn’t turn anything in and I wasn’t there.” Because I think they’ve learned that in the game, or the rules of the game, that if they just keep self-advocating for the A, that maybe we’ll somehow meet them in this, like, maybe you’re failing them and they get the A, maybe, and you’ll land at the C or something. And they see it as this negotiation practice. But more often than not, I would say I have found that, at least in my classes, that students actually are harder on themselves in their self-assessments than I would have even been.

John: And at the other extreme though, might stereotype threat play a role in some of the self-assessments as well, for students who are in marginalized groups?

Kristen: Absolutely. One, you could say the idea that many people have an inflated sense of their effort, or their knowledge of a topic, right? So we have that. And then they just hold to it. And then we have the ways in which we’ve embodied these negative stereotypes and stories about who we are as people and learners in different identity groups. And of course, that’s going to impact. That’s why the thinking about difference and how difference punctuates every part of our process is so vital, right? It’s a difference, because we all come to the table with different capacities. I think this is why, more than anything, I started asking in my self-assessments, I mean, the most important thing for me to know is, “What was most meaningful to you in this class? How are you going to build that into your life in some capacity and take it forward?” So it doesn’t mean that I’m not chasing concept understanding, it just means it’s more for me about what matters to them, and so it just changes the assessment conversation. If they can’t articulate what’s meaningful to them, that tells a pretty specific story. That’s quite different than the student that says, “I’m not such a good writer, or I had all these struggles that impacted my capacity to turn work in in this way or that way or to participate as much as I would have liked to because I have a lot of anxiety and I don’t feel comfortable speaking in front of the class. But here’s what mattered to me, here’s what I’m going to say. This has changed who I am as a person, because it changed my thinking.” That’s what I want to assess. That’s what I want to know, as a teacher.

John: And that type of metacognitive reflection, we know, helps increase learning, and there’s a lot of research to support that. So, by itself, that’s a really good practice to encourage. So we always end with the question, “What’s next?”

Kristen: So what’s next for me is I’m hoping to continue to have these conversations and continue to experiment in my classes with ways forward that I can refine what it means to do assessment and grading from a social-justice perspective. How we can best harness our communication resources—whether it be our theories, or our methods, our conversations in the classroom—to create a system that’s more just in the realm of teaching and learning. So what I’m really working on now is expanding this thing I’m calling “awareness pedagogy,” which is something I wrote about in the book in chapter eight. This process that’s built out of ungrading. Ungrading is an umbrella term that I think has been pretty widely adopted by many people to talk about a type of grading process that decouples grading from feedback. And it really focuses very heavily on learner self-assessment. In many cases, for people doing ungrading, that means that the learner themselves assigns their grade through a self-assessment process with the instructor’s support and sometimes integrating peer feedback. So, in my case, awareness pedagogy, I’m using five broad categories that I work with to help students build awareness in my classes across the kind of courses that I teach. And I teach classes in Communication, I’m doing classes on listening and interpersonal communication, intercultural communication, classes focused on conflict. And so it’s really well-suited for that discipline, which I know is different than other people that may be listening. So I’m working on that. My next project is to expand what I introduced in chapter eight into a book-length project to really get into the nuts and bolts of awareness pedagogy as its own kind of approach to social justice assessment in the classroom and what that looks like. Especially in the realm of thinking about diversity, equity, inclusion, and access, but also integrating a lot of contemplative pedagogy as part of it because that’s an area that I’m extremely attracted to in my own work.

John: Well, thank you. And it’s been great talking to you, and thank you for all of your work on behalf of students.

Judie: Thank you.

Kristen: Thank you so much. It’s been wonderful to spend this time with you all.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

213. Wicked Students

Much of the training that students receive in college involves working with well-defined problems that can be resolved using the tools and techniques of a specific discipline. In this episode, Paul Hanstedt joins us to discuss strategies that colleges can use to better prepare students to collaborate on the “wicked problems” they will face in the future.

Paul is the Director of the Houston H. Harte Center for Teaching and Learning at Washington and Lee University. He is the author of Creating Wicked Students: Designing Courses for a Complex World, General Education Essentials: A Guide for College Faculty, which is about to go into its second edition, and numerous publications related to general education and writing across the curriculum. He has worked with many colleges and universities in revising their general education requirements.

Shownotes

  • Hanstedt, P. (2018). Creating Wicked Students: Designing Courses for a Complex World. Stylus Publishing, LLC.
  • Hanstedt, P. (2012). General Education Essentials: A Guide for College Faculty. John Wiley & Sons.
  • Hanstedt, P. (2021). Helping Students Understand Our Codes: Designing Inclusive Open Curricula. AAC&U Liberal Education Blog. April 29.
  • Jessica Tinkenberg – twitter
  • Standards of Learning (SOL) – Virginia Department of Education
  • Warner, J. (2018). Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities. JHU Press.
  • Blum, S. D., & Kohn, A. (2020). Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press.
  • Hanstedt, P. (2020). Higher ed needs to redesign gen ed for the real world. Inside Higher Ed (Opinion). February 10.
  • Zull, J. E. (2002). The Art of Changing the Brain: Enriching the Practice of Teaching by Exploring the Biology of Learning. Stylus Publishing, LLC.
  • Kate McConnell
  • Textbook used at Plymouth State in a wicked problems seminar:  LeBlanc, C. (2019). Tackling Wicked Problems. Plymouth State University.
  • An article by the instructor on the course:  LeBlanc, C. (2019). What is “Tackling a Wicked Problem”? Desert of My Real Life. May 10.

Transcript

John: Much of the training that students receive in college involves working with well-defined problems that can be resolved using the tools and techniques of a specific discipline. In this episode, we examine strategies that colleges can use to better prepare students to collaborate on the “wicked problems” they will face in the future.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

Rebecca: Our guest today is Paul Hanstedt. Paul is the Director of the Houston H. Harte Center for Teaching and Learning at Washington and Lee University. He is the author of Creating Wicked Students: Designing Courses for a Complex World, General Education Essentials: A Guide for College Faculty, which is about to go into its second edition, and numerous publications related to general education and writing across the curriculum. He has worked with many colleges and universities in revising their general education requirements. Welcome, Paul.

Paul: Thank you for having me. I’m delighted to be here.

John: Today’s teas are… Paul, are you drinking tea?

Paul: I am drinking tea.

Rebecca: Yes, rejoice! [LAUGHTER]

Paul: I’m actually normally a coffee drinker, but I’m drinking tea. [LAUGHTER] I was part of the new educational developers for the POD organization and Jessica Tinklenberg sent out a care package afterwards. And in my care package was a Stash Tea Jasmine Blossom. I’d already had a 16 ounce cappuccino today, I thought I’d better keep it mellow [LAUGHTER] for later in the day.

Rebecca: Sounds nice and relaxing.

Paul: Yeah.

Rebecca: I have Yunnan Jig again. I think it’s becoming one of my new favorites. I don’t have my Golden Monkey around, so I’m going to have to drink something different. But it is also a golden-tipped tea.

John: I’m drinking a peppermint spearmint blend today… taking it easy on the caffeine.

Rebecca: I don’t know why, John, it’s halfway through the semester. What is wrong with you? [LAUGHTER]

John: I need some sleep, it’s been a stressful semester. We’ve invited you here to talk about a couple of things. We saw an article that you posted on April 29 on the AAC&U blog on “Helping Students Understand Our Codes: Designing Open Curriculum.” But we also wanted to talk to you about Creating Wicked Students. When that book came out, we both looked at it and read through it. But we were at a really early stage in the podcast, so we weren’t quite ready to ask people that we didn’t know who were already on our campus or that we were comfortable inviting because they knew of us already. And that’s long been on our list of topics that we wanted to discuss. So maybe we can start with Creating Wicked Students. What is a “wicked student”?

Paul: Good question. Maybe I should define a wicked problem first. A wicked problem originally came out of city planning. And then it was adopted by engineers because they looked at it and they said, “This is exactly what we’re dealing with all the time.” A “wicked problem” is a problem where the dynamics are in flux, they’re shifting. What the problem looks like on Tuesday and what the problem looks like a week from Thursday can be completely different. Oftentimes, the data is incomplete. Oftentimes, previous solutions don’t apply. Very often, wicked problems are problems that must be solved, you can’t let it sit. There’s oftentimes contention about how to solve it, how to fix it. And almost always—and this is where the gen-ed person in me gets all excited—the solutions for a wicked problem are going to require drawing from a bunch of different fields. So the best example, and it’s a horrible example, but everyone will understand it immediately, is COVID. COVID is the perfect, horrible, wicked problem. The dynamics have been changing constantly. Even now, we’re still waiting to get around the bend and say, “Oh, okay now we’ve arrived.” If it were purely a science problem we’d be in great shape because we would have been done early this summer. But there’s politics in play, there’s economics in play, there’s culture in play, there’s religion in play. There’s messaging and communication and images and memes and social media and technology. All of these things are creating a dynamic quality to COVID that makes resolving it very, very difficult. And, of course, we must resolve it. So that’s a wicked problem. A wicked student is a person who can solve that problem [LAUGHTER] when they graduate from college. But in engineering, what they said when they saw wicked problems is, “Our students need wicked competencies, because they’re going to face, once they leave university, they’re going to face these wicked problems in the field.” So much of what we do in education is tame, static, the answer is at the back of the book, there’s clarity. So a wicked student is a student who can move beyond simple answers, move beyond solving problems where they know the answer because they’ve been told the answer, or they know the answer because it’s at the back of the book, or they know the answer because they apply an algorithm and arrive at a solution that’s clear. A wicked student is a student who can deal with uncertainty, who can struggle, who can ask good questions, who can draw from different fields, who can collaborate well, and who can, when they fail, pause, step back, deliberate, reconsider, and try again. I have a colleague in biochemistry, Kyle Friend, who talks about what he wants his students to finally know is that he gets it wrong 70% of the time, and that’s normal. For so many of our students, getting it wrong is bad, it’s failure. And, frankly, that’s our fault because we create that dynamic.

John: Would you say that most curricula is well designed to create wicked students who are able to handle these problems that are not so well defined, that cross disciplinary boundaries?

Paul: No. [LAUGHTER]

Rebecca: The end. [LAUGHTER]

Paul: Yeah. So many of the systems that we’ve put in place—in Virginia, it’s the Standards of Learning tests—they ask for certainty and clarity. I’m running a writing pedagogy seminar on my campus right now and using John Warner’s wonderful book Why They Can’t Write. And he makes a point that the five-paragraph theme, which for those of us with composition and rhetoric backgrounds, is the root of all evil. He says, “Actually no, it’s just the tip of the iceberg. It’s the symptom of the larger problem.” And the larger problem is our desire to mass produce education, to make education easy, to make our grading easy, to make learning easy so students can feel like they’ve mastered it, because they’ve got the five facts, or the 10 facts, or the 11 dates, or the four algorithms that they need to understand in order to do well in the class. So no, I think so much of what we do is tame.

Rebecca: So if we’re saying, “Yes, please,” to these wicked students, what do we need to do in our classrooms? What can we, as faculty, implement in our course structures?

Paul: Yeah, how long you got? [LAUGHTER] So…

Rebecca: I have all day, if you’ve got all the answers!

John: And there’s a good book on that, too, that people can refer to.

Paul: There is a good book on that, yeah. So any number of things. One is finding ways to bring uncertainty, lack of clarity, into the rhythm of the classes that we teach. Years ago, I was running a workshop and a gentleman named Dan Clark who was then at Western Oregon University, he’s not there anymore, and I forget the institution he’s at. But he talked about Monday morning riddles. You know, you walk in, and whatever field you’re in—whether it be psychology, or accounting, or politics—give a problem and put students in groups and have them not come up with one solution but come up with three solutions, three ways to approach it. And ask them to be able to articulate why. And then put them all on the board and discuss them. And the point is not: there is one answer, and there’s one way to get to that answer. And oftentimes there is, let’s recognize that. But learn to play, learn to ideate, learn to work with multiple paths, because sometimes the first path you take isn’t going to work. And I should say, really quickly, I’m always aware that there are people in the room in various fields, whether it be French or chemistry, where they’re saying, “Well, there are facts, and they need to know the facts.” Yes, it’s not about knowing the facts, it’s about whether or not 100% of our time in the class is about content delivery and content reiteration, or if we create space in there for application of that content, and sometimes application of that content and then discussion about that application, where clarity and certainty isn’t necessarily achieved because that’s not the point. So it can be in our daily practices, we can create exam questions like that. I can, in a literature class, give students a poem and say, “We’ve been studying the Romantic poets, which one of the Romantic poets wrote this poem?” The fact that they’ve never seen the poem isn’t the point. Actually, I might choose to include a poem from a poet they’ve never read. In fact, I might include a poem from a poet who’s not even a Romantic. Thing is, can they analyze? Can they think? Can they explore? Are they being deliberate about their exploration? And what I’m grading them on is not their ability to get to the answer, it’s their ability to travel to make that journey. I could keep going if you want me to. [LAUGHTER] We can talk about paper assignments, we can talk about text selection, we can talk about the goals we create for a course. When do our goals move beyond content delivery? When do our goals move beyond application? When do our goals move into our ideals? I mean, I didn’t go to grad school and then take a job where I was diving underwater for nine months a year in order to just make sure that students could identify passages from Dickens that we’d already discussed. There’s more to it than that.

Rebecca: One of the things that I really love about Wicked Students is, it’s a value system that we want students to embody. And you’ve talked about one way of getting at that and offering space for these messier problems or the journey. What are some other ways within the structure—thinking about syllabus, learning outcomes, assignment design—that might continue to embody those values? It’s one thing to say, like, “We want to do that.” But what does it look like when we’re actually doing it?

Paul: Right. Well one of the things, and everybody’s doing it these days but why not, here’s my nod to Susan Blum [LAUGHTER] and her book Ungrading. One of the things, if we’re going to have this uncertainty, if we’re going to ask students to take intellectual risks, if we’re going to ask students to not freak out when they don’t arrive at the perfect answer, we need to be sure that in our syllabus, and in the day-to-day work of our course, there are plenty of opportunities where the grade is not of consequence, where they are not hurt by the ability to get it wrong. So I talk about ungraded work that happens in the class, I talk about minimally graded work that might happen outside of the class, proportionally graded work where the first time they do it, it’s only worth 2%, and the last time they do it, it’s worth 20%. But they’ve progressed and they’ve gotten feedback. So that would be another one of the ways, even in things like large lectures, the pause… you’re 20 minutes in, you’ve been delivering content to 270 students, and now you pause. And maybe you use clickers, maybe you use PollEverywhere. Maybe you simply put that question up on the screen and say, “So how might this apply here? What do we think? What are the ways to approach it? This is considered a difficult problem, how would you deal with it?” Maybe you simply say to them, “Pair up with the person next to you or jot some notes, think-pair-share, with the person next to you.” And then you just do a little bit of pointing to the room, “You up there in the red sweater, let me know what you’re hearing. Keep in mind that we’re not looking for the perfect answer, we’re looking for thoughtful answers.” So those would be two ways. There’s a lot of other ways. I mean, I like the idea of co-writing our goals with our students. And letting them know upfront that, with this class, it is not about static information, it’s about construction of knowledge, construction of ideas. The goal here finally is, not just for me to tell you what I know or what other people think, but for all of us to approach this material, whether it’s in literature, or in chemistry, so that we take the thinking just a little bit further, or maybe a lot further.

John: How can we sell this to our students, especially our freshmen students who come in from a curriculum that’s very structured, that’s very well defined by disciplinary boundaries, and so forth, and where all the problems have nice, neat, easy solutions?

Paul: And oftentimes, they’ve gotten into university because they’ve been really good at playing that traditional game, right? And here we are changing the rules. [LAUGHTER] Well, part of it is you need to create spaces. You can’t just give them a big assignment at the end of the semester that’s wicked, or a test question that’s wicked, without creating those spaces for them to experiment, fail, and get used to it. But what I like to, frankly, do is walk into class on the very first day, and sometimes… I like to be very dramatic… and I don’t even give my name. [LAUGHTER] I just put that painting up on the screen, and I ask that question that can’t necessarily be answered perfectly, and then I ask them to write. I was at a workshop a couple years ago at a college up in Maryland, where a meteorology professor talked about on the very first day, he hands his students a meteorology map. He says, “Tell me what’s going to happen next? What’s the weather going to be like?” And he goes, “They get it wrong 97% of the time, but that’s not the point. The point is for them to work with it and play with it. And then as the semester goes along, you can periodically hand that back to them, and they can see themselves making progress.” So I think the first step is to begin. So often we try to just say, “If I just repeat it, then they’ll get it.” It’s hard to use words to combat 18 years of experience and 13 years of educational experience. We need to create alternative experiences where they live, where they see it, where they understand.

Rebecca: I know one of the things that we emphasize in design is process over product, often. We have some students who can make a fantastic product, but they do no process.

Paul: Mm-hmm.

Rebecca: And so it always ends up being a really long conversation about, “Well, what was your journey here? Why are you doing that? How does that meet the creative brief? Or how does that meet the problem? How does that solve the problem?”

Paul: Right.

Rebecca: It just looks good… it doesn’t mean that it does anything. So I think we’re always working on ways to emphasize that process piece more, because it tends to get overlooked. It’s like, here’s the beginning, here’s the end, but that muddy, messy middle part—which is the most important part, especially to develop wicked students—is often overlooked.

Paul: I really do like multiplicity ideation, multiple iterations. And maybe it just happens in the planning stage where you say, “Rather than just going into the task…” because there’s very few fields where you just jump into the task… “come up with plan, give me three different plans, three different ways to approach this, three different ways we might design this, three different ways we might interpret this poem, three different ways that we might design this experiment, three different ways that we might think about how to approach culture in rural Germany, three different ways about how to solve this medical problem.”

Rebecca: So three is really important, Paul? [LAUGHTER]

Paul: Three, yeah. [LAUGHTER] I guess I’m sort of focusing on three. Well, I mean, there is a certain sweetness to three. Five is way too many oftentimes, and two is probably not enough. So yeah, I don’t know why I got on three. [LAUGHTER] But thank you for teasing me about that, I appreciate that actually. [LAUGHTER]

John: And as you move through the course, you also suggest that the degree of complexity should be adjusted from the beginning to the end of the term. Could you talk about some ways in which you might do that in a given class?

Paul: Sure, yeah. So the classic example, and I use it all the time, comes from my colleague, Chris Connors, who’s in geology here at W&L. And he talks about how early in the geology course, they’ll provide a rock sample to students. And they’re asking for, in a simplified form, recommendations about drilling or mining. And in the early data sets, there will be one or two pretty clear paths, and a minimum of noise, being data that is meaningless or simply distracting. And if the student’s been paying attention in those early data sets, they should be able to get it, they should be fine. Later data sets, middle data sets, they’re more complex, more noise, and a multiplicity of reasonable answers. Some of which are going to be more self evident than others, and some of which need to be constructed. But the students, if they’ve been growing, they should be okay. And part of what they need to do is not just say, “This is the recommendation I’m giving,” but, “Here’s the why this recommendation, rather than these recommendations.” Latter data sets are just wide open, lots of noise. And the paths of recommendations, the ideas, the thinking, the conclusions that are being drawn, really have to be constructed almost from scratch by the students individually. And so that’s one example of how progression could occur. Also worth noting that you can increase the degree of difficulty by the level of interaction that you have with students: the amount of coaching that you offer, the number of tips or hints that you provide, how much you’re walking around the room, the degree to which you’re having them collaborate versus doing it individually. What I love about the geoscience example, too, is that in many ways it’s, again, as someone who works in literature, that’s what we do in a literature classroom. When I’m teaching Victorian literature, I begin with a relatively simple Victorian text. And then by the end of it, I’m handing them Eliot, which is millions of words, and a lot of noise. And they’re having to build these maps, construct these meanings. And I’ll just point out that construction of meaning, that production component where the student is taking it just a step further, they’re not handing to me a reading of Middlemarch that they’ve read somewhere else. They are building a path, they are collecting the data, they are doing that work. And finally, that’s what we want. If we’re going to graduate these students into this complex, messy, messed-up world, we had better have people who can build that path thoughtfully and deliberately themselves. Otherwise, we end up with a situation where some large portion of our nation might be being misled by other people, hypothetically.

Rebecca: Definitely hypothetically.

Paul: Yes.

John: One of the barriers though, I think, is that our classes and majors and programs tend to be structured very much within very narrow disciplinary boundaries. Would it help if, perhaps, we had more interdisciplinary courses, and activities, and seminar courses, in most colleges?

Paul: Yes. And I don’t want to take us down a rabbit hole unless you want to go with me [LAUGHTER] of general education, and over the last 40 years, the movement has been away from distributional models that say, “Two of math and science and two of social sciences and two of arts and humanities.” Distributions still exists, but the movement has been towards more integration, more blending, more of a recognition that it’s not about what content we’re delivering, but how that content connects to other content. How, when I teach a class on artistic and literary responses to science and technology, I’ll have a student reading Coleridge’s Eolian Harp and giving me an analysis that ties it to string theory in physics. How do we make connections? How do things blend? So, sometimes, yes, majors need to do what majors need to do, and I want to come back to that in just a moment. But gen ed can do a better job of just replicating what majors are doing. Gen ed, it shouldn’t be just about exposure, it should be about synthesis, it should be about reflection, it should be about building something, making connections. And gen ed is going a step further and really looking at the way high-impact practices, for instance, rather than having models that are driven by distribution, they can be driven by high-impact practices: e-portfolios, which is about synthesis, study abroad, internships, undergraduate research, first-year experiences, senior capstones, all of these things that are synthetic. They’re asking students to make these connections. They also push students towards agency, towards really doing the work themselves. A student abroad, one of the reasons that’s a high-impact practice is because from dawn to dusk, they have to cope. Just leaving their dorm room, leaving their dorm, is a complex act. Getting lunch is a difficult, challenging thing where they have to assume agency and do it. So right now, though, I just also want to step back and talk about majors because I referenced that earlier. Let’s face it, increasingly, majors are becoming interdisciplinary. They’re either doing it by building interdisciplinary majors—biochem, gender studies, environmental studies—but even within majors, they’re recognizing the ways that they need to bring things into play. I mean, psychology has shifted from being a social science to more of a hard science, because it’s bringing neuroscience into play. Again, there’s this idea that the world is a messy place. The academy, we’ve got our little hallways and our little cells and our little silos and our little blocks that we built, those are very convenient for us. They’re not necessarily helpful for the students, there’s some convenience and some value to it for the students. But if that’s all they experience, and if that’s all they see, and if we allow ourselves to fall into that, I don’t know that it helps.

Rebecca: So one of the things that often is a hard sell to students is general education.

Paul: Yeah, [LAUGHTER] I’ve noticed that!

Rebecca: It’s advising time around here, so helping students identify things outside of their major that maybe they might be interested in exploring or might be useful to them in these wicked problems as they leave the institution and go into the profession. What’s some advice that you might have in making that sell towards general education? Is part of the issue the way our general education is structured, so it’s not a good sell? Or is it partly just communication?

Paul: Yeah I’ll deal with the communication question first. It is definitely communication, we spend a lot of time on the “what,” and very, very, very little time on the “why.” I mean, in so many places, if it’s not literally a box check, it’ll have a brief description of the humanities, and then it’ll be on to just a list of courses. And frankly, I would say that if you’ve got a particular learning outcome or goal for general education that you want students to meet, and you’re trying to impress them with how important this idea, this value, this goal is, and then you’ve got 700 courses that can fulfill that requirement. I’m sorry, that’s a mixed message because it basically says, “This goal is incredibly important, so important that anything can fulfill it.” And again, oftentimes what’s happening there is the system is set up to please the department or the anxieties that departments might have or to get butts in the seat. So figuring out how to better communicate that “why,” both on the front end and in advising. We have ready answers for why students should take particular courses in our field. Do we have ready answers for why a poet should take a mathematics course? For why a student in sociology should study dance or yoga? We need to work on that. There is nothing wrong with talking to your colleagues and coming up with some answers so that when students come into play, you can sell it convincingly. The other question about the structure, I definitely think that that’s a factor. If we’re selling them isolated little building blocks, it communicates to them what matters and what doesn’t matter, why it’s happening, why it’s not happening. You know, they get distribution in high school, and call me naive or idealistic, but I think even the most cynical, jaded, least-engaged student in the world comes to college and wants something different. They want it to feel like a challenge, that they’re moving up, that they’re going somewhere else, this is something more. And if we don’t give them that, then they come away with that idea that college is just a transactional thing. That’s a failure.

John: I think this brings us quite naturally to your blog post on open gen ed requirements. Could you tell us a little bit about the concerns you raised about having a completely open or very open gen ed curriculum?

Paul: Sure, full disclosure—and that blog was edited to fit a certain word count, this full disclosure was in the original version—my son goes to an institution with an open curriculum. I actually think… and I’m not going to name the institution, I’m tempted but I’m not going to… I actually think this institution does an extraordinarily good job of advising. So that students, no matter what they’re studying, are pushed and encouraged to go into and take courses in different fields. So it can work really well. A couple of weeks ago, my daughter and I were visiting colleges, she was someplace where the guy ended the tour by saying he was there to study economics, he said, “If you told me a year ago, I’d be taking two classes in classics and really enjoying them, I would have laughed in your face.” That’s a win, that’s a well-done job. So open curriculums can work and I want to be careful not to just dismiss them completely. Part of the issue with them, on the faculty side of things, I think, it’s a missed opportunity because it means that faculty from different disciplines in different departments don’t need to be in conversation with each other. I think that’s a loss. And that’s unfortunate, because the more we talk to each other, the better our teaching all-around is going to be. But really, for me, the major concern has to do with students. Not all students come into college recognizing the value of that liberal arts component, that liberal arts dimension that is so much a part of the American educational system and has been almost from the start. My son grew up in a household where we went to Hong Kong for a year to help them revise their university curriculum from a three-year system with just a major to a four-year system that included liberal arts and general education. That’s been written into his DNA, written into his family history. We talk about stuff like this at the dinner table because I’m such a geek. He has an advantage because he walks in and even if he is really interested in studying physics, he knows that there’s a value to taking German courses and philosophy courses and courses in art history. So for him, in an open curriculum, there are not going to be any signposts. As bad as a really highly-structured curriculum might be, there are a lot of signposts, “This matters, this matters, you need to do this, you need to do this.” In an open curriculum with no signposts, the student who has that level of knowledge, that insider knowledge, they’re going to be okay. The student who walks into that setting, for whatever reason, without that knowledge, maybe it’s because they’re from an immigrant family that isn’t familiar with a tradition of the liberal arts, that phrase doesn’t even necessarily mean anything to them. Maybe they’re from a first-generation family. A young woman that I know actually came from a family where her parents had both gone to college, but they were running a business, and that was really their focus, a nd so this idea of the liberal arts and the value of liberal arts just wasn’t part of the conversation that they had. Perhaps it’s because students come from a historically or structurally marginalized background, I don’t know. And of course, we want to be careful about essentializing about any groups. But the fact of the matter is, if we have an open curriculum, and we don’t have some guardrails in place, we’re not paying attention, it can be inequitable. Certain students are going to benefit powerfully, other students are not going to benefit as much. That’s wrong. We’re educational systems, we’re educational institutions, our goal is to progress everybody forward, not just to allow the privileged to continue to be privileged, and the less-privileged to continue to be less privileged.

John: A lot of first-gen students come in with a very specific career goal and a very narrow definition of what courses they think are going to be relevant for that, not recognizing the wicked nature of some of the problems they’ll be facing or the world in which they’ll be living in a few years. How can we build a system that would correct for that if there is going to be a more open curriculum?

Paul: Yeah, well, I’ve already mentioned advising. So part of it is we need to have, as an institution, and as instructors and colleagues, a language of practice, of shared practice, shared understanding, shared values, that we communicate to the students. And one would hope that that happens not just in the advising session, because that’s really kind of too late, but in everything—from the initial literature the student received, to the alums and the alumni stories that we tell in the alumni magazine or the people that we bring to campus to have conversations with our students, the internships we set up with alums, and all of those kinds of things. So that’s one thing, advising, but also advising writ large. I also think we need to create some spaces where reflection and integration is going on. So maybe it’s a completely open curriculum, but maybe in order to sign up for your classes next semester, or maybe in order to complete your work as a sophomore, or as a junior, or you do your senior thesis, you have to have a reflective component that kind of puts some of the pieces together, that does that sort of integration, that synthesis. Students, they’re so busy, and they’re doing so many things. Sometimes we can reflect, but oftentimes we’re just doing. And so where do we build those moments to make sense? James Zull in The Art of Changing the Brain talks about how there are educational experiences, and they’re really just data until the students integrate it and reflect on it. And at that point, they move from data to knowledge, meaningful knowledge. So where do we create those spaces to make meaning out of the experiences that they’re having? Kate McConnell, AAC&U, says, “Assess and then disaggregate your data.” So those are three things you can do: advising, reflection, assessment and disaggregation. Finally, when I talked about this in the essay, and this is one of my favourite metaphors, my colleague, Rich Grant—who’s a physicist at Roanoke College and he’s actually now the acting Dean of the college—he talks about how students come to college with all these experiences, some personal, some educational, some social, some related to social media, and they’ve got this box of puzzle pieces. And again, on some level, they want meaning, they want purpose, they want to know how all these pieces fit together. If all we’re doing in college is just dumping more puzzle pieces into the box, I don’t think that’s success. Some students will thrive with that, other students will not. If our goal is to make sure that everybody thrives, what can we do that’s deliberate and thoughtful?

Rebecca: I was reflecting a little bit on what you asked, John, and then also the way that you’ve been talking about gen ed, and wicked problems, Paul, and I was thinking: I was a first-gen student who didn’t give a crap about gen ed. I remember. I had a couple things I was interested in, so I curated in a way that followed my interests, but I really didn’t see a lot of value in it until later on, when I had an opportunity, really, to work on a messy problem, to work on a wicked problem. And that was in a class that was cross-listed between geography and architecture…

Paul: There you go.

Rebecca: …and worked with community organizations. And then all of a sudden, I was like, “Oh, I need to learn all of these things.” Like then I had like a to-do list of things I needed to learn.

Paul: Yeah, and so they did a great job there of bringing the messiness of life into the classroom. I’ve got a friend who’s an architect, and a very good one, and every time he moves up the ladder in his organization, the problems get messier and messier and messier. And almost from the second stage, he wasn’t prepared for it. So how do we bring that messiness into the classroom, into the academy, into the curriculum? That’s a great example, I love that.

Rebecca: It’s making me think, like, having a really messy problem really early in a college education would be really important to getting students on board with the idea that you need to learn a lot of things…

Paul: Mm-hmm. [LAUGHTER]

Rebecca: …to have a lot of different kinds of experiences.

Paul: And there are increasingly places that are doing that. I believe Plymouth State is. In the first year, they’ve got a seminar that deals with a messy problem. In the Netherlands, almost all of the university colleges, the small liberal arts colleges and the larger universities, have problem-based and project-based curricula for gen ed, where students are taking these things on from the beginning.

Rebecca: That’s interesting, too, because that means a gen ed class has been designed as a gen ed class from the start, and not necessarily a major class that slots into a category.

Paul: Yeah. In gen ed reform, really, one of the stickiest questions is, “How do we determine what counts as a gen ed class or not?” And in some institutions, if it’s a gen ed class, it has a gen ed prefix and designator, and it is a gen ed class and nothing but. And, on the other extreme is, “Well, anything can count as gen ed.” And then, in between, you’ve got models that have core courses and distribution courses. And the core courses are gen ed specific, regardless… And the line that I constantly give to gen ed committees as they’re doing revisions is, “If you’ve got gen ed goals, you have structure, an apparatus, a process, a protocol in your course approval that honors those goals, unapologetically.” And that’s hard, because you’ve got departments pressuring you. But if we’re going to do it, we have to do it, we can’t just give it lip service and let whatever count.

John: Early in my career at Oswego, I spent one semester serving on our faculty assembly when we were discussing putting together the final stages of a gen ed curriculum. And much of the discussion was basically departments being very worried about losing FTEs, losing lines, and so forth. How can we get past that in these discussions?

Paul: Yeah. One of the things is, in the design process, and in the communication process, about whatever new gen ed models you’re bringing forward for conversation, you have to foreground that there are generally going to be multiple paths into the curriculum. So in the distribution model, well the history department has their own, that’s great because it’s clear, and it’s simple, and there’s assurance, and every department has their own, so everybody’s happy. If you’re going to make an integrative model that’s going to counter that, you need to show the history department that, look, in this model you have your one course. In this model, you can enter it the first year, which is really going to appeal to some people who prefer working with first-year students. You can enter at the capstone level, the senior or junior year, which is really going to appeal to some people who enjoy doing that. Or there’s these interdisciplinary courses over here, which is going to appeal to some faculty who enjoy doing that. Or there’s these hobbyhorse courses over here, that might really be interesting for people who have never been able to teach the bizarre, esoteric topic of their dissertation in a gen ed course, but now you can. And the beauty of that last one, in particular, my colleague, Hedley Freake who is retired from the University of Connecticut, he’s a chemist and he says, “If you give me an option between teaching Chem 101 yet again, or teaching gender and nutrition in developing nations, I’m going to take that latter course all the time, because it’s more engaging to me, it’s more engaging to students. And this is shocking, but the more engaged students are, the happier they are, and the happier I am.” But you have to really be deliberate about foregrounding those multiple entries and the benefits of those multiple paths. Again, it’s the simplicity of the algorithm, and all the experience we have with that safe, reassuring algorithm versus, “Oh, this is new, and it’s different.” And how do we clarify that idea with more than just words? We just had this conversation on my campus yesterday.

Rebecca: Sounds like a wicked problem to me.

Paul: It is, education is a wicked problem! [LAUGHTER] And it’s terrible, right? Because I was thinking, it’s not because education’s the solution. But clearly, we’ve got podcasts, we’ve got conferences, we’ve got discussions. It’s a problem, it’s a challenge, it’s fluid, it’s dynamic. It’s changing constantly, the opportunities, the challenges, it’s all… Gosh, you teach your 9 o’clock class and it goes perfectly, you teach the same thing at 11 o’clock and it’s a disaster. I mean, you know… [LAUGHTER] Why? Because students are wicked. [LAUGHTER] There’s a chemistry there that we can’t control, a dynamic that we have to work with. I love that, and this is it. I mean, wicked problems are engaging. They engage us emotionally, intellectually, socially. They’re exciting, the brain reacts to them, we like puzzles. The part of the brain that wants to solve, that appeals. And I do think that that, again, if we’re thinking about changing the way students think about education. If education is static—you hand me this and I hand it back—that’s disengaging. And a certain number of our students are not going to see the value of that. If, on the other hand, we say, “Here’s a problem, can you solve it?” That’s powerful.

Rebecca: And it’s fun.

Paul: It’s fun.

John: For them and for us.

Paul: Yes.

Rebecca: Dare I say education is fun? [LAUGHTER]

Paul: Wickedly so. [LAUGHTER]

Rebecca: So we always wrap up by asking the big, huge question, “What’s next?”

Paul: What’s next? I think we’re all still recovering. I mean, of course we are. But even those of us who haven’t been ill, the brain, our sort of cognitive power, is still depleted. So naps, naps are next. I’m very excited. I mean, I just got the contract for the revision of the General Education Essentials book last week. And I’m just delighted to be able to take that on because it came out in 2012, I wrote it in 2011, and in those 11 years my thinking has really evolved. And so I’m looking forward to seeing where that goes. I am always with the wicked pedagogies and the wicked problems thinking about, when I do workshops or something, I’m always collecting or trying to collect the examples that people use. So I kind of think, ‘Where can I go with that? Is there another way to approach this? Are there different modes of communicating that can be brought into play?’ I don’t know. So recovery and exploration and maybe trying to find some of the joy that was threatened, if not, depleted.

Rebecca: Yes. More joy, please. [LAUGHTER]

Paul: Yes please. And chocolate.

Rebecca: Yeah. It’s a good side.

Paul: It is, yeah.

Rebecca: Well thanks so much, Paul. We really enjoyed having this conversation and you always bring great ideas to the table.

Paul: Well, thank you. I very much enjoyed chatting with you.

John: We’re really happy that you’ve joined us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

212. Faculty Mindset

Research on the impact of mindset has often centered on the mindset of the student. In this episode, Elizabeth Canning joins us to discuss the impact that faculty mindset has on student achievement. Elizabeth is an Assistant Professor in the Department of Psychology at Washington State University. Her research focuses on how to create equitable and inclusive instructional environments.

Shownotes

Transcript

John: Research on the impact of mindset has often centered on the mindset of the student. In this episode, we discuss the impact faculty mindset has on student achievement.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners..

[MUSIC]

John: Our guest today is Elizabeth Canning. Elizabeth is an Assistant Professor in the Department of Psychology at Washington State University. Her research focuses on how to create equitable and inclusive instructional environments. Welcome, Elizabeth.

Elizabeth: Thank you.

Rebecca: Today’s teas are… Elizabeth, are you drinking tea?

Elizabeth: I’m drinking water today.

Rebecca: Alright, still a good choice. The base of tea, of course.

Elizabeth: Yeah. [LAUGHTER]

John: And I have a restricted set of tea because I brought most of my tea back up to the office, but I’ve been sent home with COVID. So I do have a ginger peach black tea still here though.

Rebecca: That sounds like a pretty standard fare.

John: It is.

Rebecca: I made it out to that tea shop I’ve discovered. And so I have a new one called Yunnan Jig…

Elizabeth: Ooh!

Rebecca: …and it’s a golden-tipped black tea.

Elizabeth: That sounds delicious.

Rebecca: It is very delicious.

Elizabeth: Exotic teas.

John: We’ve invited you here today to talk about your 2019 study that examined the effect of instructor mindset on racial achievement gaps in STEM disciplines. Could you tell us a little bit about this study?

Elizabeth: Yeah. So this study was a study that we did while I was a postdoc at Indiana. It was in collaboration with Dr. Mary Murphy, Katie Muenks, and Dorraine Green. We were really interested in instructor beliefs about intelligence… so whether they believe that intelligence is something that is innate, something that you’re just born with, you can’t change it very much, we call that a fixed mindset. And what we also call a growth mindset is this belief that intelligence can grow over time and change with effort, strategies, help-seeking, things like that. And we were interested in whether instructors’ beliefs about intelligence then predicted the experiences that students have in their classes, and then how well they do in their classes. So we sent a survey out to the whole university, and we didn’t expect a whole lot of responses because faculty are busy and all of that. But we actually got a lot of responses. We had 150 STEM faculty respond to our survey. And so with that, we were able to link their survey responses to the grades in their courses over a two-year period. And we ended up with this fantastic rich data set that we were able to look at students’ grades in their courses.

Rebecca: How prevalent was the fixed mindset amongst the faculty that you surveyed?

Elizabeth: Yeah. So it was pretty normally distributed, shockingly so I think. A lot of people think that fixed mindsets might be more rare. Other people think fixed mindset might be extremely prevalent, but it was pretty normally distributed. It’s a continuous scale so if you graph it, it’s a pretty normal curve. So we’ve got the most people in the middle. And then we’ve got some people at each extreme. But what’s interesting when you look at it, though, is especially on this scale, most people, they won’t go extreme, like the very, very tip of the fixed mindset scale. But they’ll do the next level over. It’s like, “I’m not gonna be that person, but I’ll be right next to it.” So it’s interesting to think about that when you think about the profession of being an instructor, and part of your job is to educate people. But yet we’re finding a significant amount of people have this mindset that intelligence or abilities can’t change.

Rebecca: So you talked about it being a normal distribution, was there any variation amongst race or gender, ethnicity, age, or STEM discipline?

Elizabeth: Yeah. So we get that question a lot. Like, can we predict who it is that has the fixed mindset? Well, this is the only table I’ve ever published that had every single line in it non-significant. [LAUGHTER] So we tested pretty much everything in our data set—whether mindset differed by gender, by faculty race, by age, by teaching experience, tenure status, anything that we had in our data set—and we found no differences by mindset. So it seems like having a fixed mindset or having a growth mindset is prevalent among all faculty regardless of these characteristics. We also looked within departments because a lot of people want to think that it’s, “Oh, it’s those economists or…” [LAUGHTER] looking at you, John. Or, “It’s those computer scientists, or the physicists, or the mathematicians.” But we didn’t find any differences by discipline. So faculty in any kind of discipline can endorse a fixed mindset or a growth mindset. And this suggests that these mindsets are not going to resolve itself on its own. Like if this was a generational thing, then we would expect it to resolve over time, or we could go into a specific discipline and educate faculty, but it seems to be pretty widespread, regardless of these characteristics.

John: That was one of the most surprising things in your study, to me. I was expecting that this would vary, particularly with age, but also perhaps with gender as well. And I was thinking that maybe this would be better over time. One of my favorite quotes from Paul Samuelson, an economist who died a while back, was, “funeral by funeral, the science makes progress,” and I was kind of hoping that that might occur with growth mindsets here, too. But it doesn’t look like that’s going to happen.

Rebecca: So we’re left with saying, “Great…” [LAUGHTER]

John: When you were looking at the effect of instructor mindset on student grades, were you also controlling for the characteristics of the students as well?

Elizabeth: Yes. So we controlled for a number of things in our analyses, we wanted to put anything in the model that might predict student performance. So we controlled for whether the student was the first in their family to go to college, their first-generation status. We controlled for their prior achievement level, so their prior GPA before they enrolled in that class. We also controlled for a number of course characteristics. So whether the course was, at this university, a 100 level, 200, 300. So, like, an entry-level class versus more of an upper-division class or upper-level class. We controlled for a number of faculty characteristics as well, like whether they were tenured, their age, how long they had been teaching. And so all of that was in the model to control for those variations in terms of what predicts their grades. Because a number of things we know predict grades, like class size. If it’s a bigger class versus a smaller class, we know that’s a pretty robust finding. And so we controlled for that in all of the analyses.

John: What was the overall effect, controlling for all the other student and instructor characteristics, of instructor mindset on student grades?

Elizabeth: So students, on average, received a higher grade in faculty’s classes where they endorsed more of a growth mindset. And this was, again, controlling for all of those things, regardless of student characteristics, class characteristics, and faculty characteristics. We also looked at this by student race, so we found an interaction with student race. So it’s not just that everybody on average is receiving or earning higher grades in the growth-mindset courses. This is particularly true for students with racial- ethnic minority status. So Black, Hispanic, Native American students performed better in the courses that had faculty who endorsed more of a growth mindset. When we look at the achievement gap between White and Asian students compared to Black, Hispanic, and Native American students, we see that this racial achievement gap is twice as large when the faculty endorsed more of a fixed mindset, compared to when they endorsed more of a growth mindset.

Rebecca: Can we talk a little bit about differences in assessments or the way that courses are structured between the fixed-mindset faculty courses versus the growth-mindset courses, because I think your paper talked a little bit about that as well, right, Elizabeth?

Elizabeth: We couldn’t dig into the specifics of it just because of the scale of this project, but we’ve done a lot of follow-up to see: What is it that faculty are doing in these classes? Because students are obviously picking up on it. It’s not just the belief that you hold near and dear to your heart, this is something that is being communicated to students in some way. And what we know from our other research is that it’s communicated in a lot of ways depending on the instructor, depending on the class. So it can be communicated in your course policies, like how you design your syllabus, how many assessments you give in your class. So, fixed-mindset professors are more likely to have a midterm and a final, and that’s your only opportunity to display your abilities in that class. Whereas the growth-mindset professors are more likely to have weekly quizzes where you can improve over time and see that improvement over time. And mistakes are less deadly in those classes, so to speak. So it’s in the way they design their courses, but it’s also in subtle ways. So what they say in class, how they talk to students who are struggling in their office hours, it’s in their attitude, it’s in a lot of different behavior. And students are pretty perceptive, they can pick up on it pretty quickly.

John: So we can significantly reduce racial achievement gaps if instructors have growth mindsets. Is instructor mindset something that’s changeable?

Elizabeth: Yeah, I think so. We know a lot from trying to change students’ mindset beliefs, it’s actually pretty malleable. You can teach people the science behind how our abilities grow over time, the changes that occur in your brain in terms of neuroplasticity, and faculty are pretty receptive to that. Most faculty want to do things that are going to benefit their students, most faculty want to be good instructors. And so it might just be finding ways to communicate that. In some of our research, we’ve found a disconnect between what faculty think their mindset is and how they’re communicating that, to what students are actually perceiving. And so it might just be communication, making sure that you’re very explicit about what you believe in your class, standing up on the first day and saying, “Here’s what I believe about abilities and intelligence, you don’t have to be, quote-unquote, ‘smart’ to do well in this class. Here are other ways to do well in this class, like learning and improving and using different strategies and things like that.” We also know that there are very critical times where these messages matter more. So there are times during the semester where students are searching for information about their abilities. So when you hand back that first exam grade, or that first assessment that you give in your class, that’s the perfect time to communicate your mindset beliefs, particularly if they’re a growth mindset. Because students are searching, “What does this mean? Does this mean that I’m good at this? Does this mean that I’m not good at this? Am I going to do well, should I drop this course?” It’s a time of uncertainty for students. And so, for faculty to communicate those beliefs during that critical time, it can set forth sort of the snowball effect for how they should view their learning and improvement throughout the semester. There’s a number of ways that faculty can do this in their classes. But back to your original question about, “Can we change faculties’ mindset?” I think so. I think there’s a lot of literature suggesting that we can do that.

Rebecca: And there’s been a lot of money pumped into making these STEM pipelines in the first place. We want them to be effective. So investing in this education around mindset and learning might be a really good use of funds.

Elizabeth: Yeah, one of the things that I think was really shocking about this paper is the faculty that reported their mindset beliefs, there were 150 of them. But when you look at all of the students that they touch, over a two-year period, how many people they teach, it becomes a really big number pretty quickly. So in this sample it was around 15,000 students that these 150 faculty taught over a two-year period. And so, instead of intervening with 15,000 students, you might intervene with 150 people and see similar or maybe even greater effect.

Rebecca: I know from our experience, John, with working with faculty around mindset and around helping students learn how they learn, that faculty who demonstrate a growth mindset are often very willing to share what they’ve learned about learning with their students. And so having an intervention with 150 people then reaches many of those students, because that information ends up being communicated out in a more distributed way.

John: One thing I’m wondering is whether you can separate out, in these results, the impacts of the way in which people teach from the messaging that’s coming up indirectly in other ways? It strikes me that that may not be possible, in that the instructors with a growth mindset provide lots of opportunities for students to make mistakes and learn from those mistakes, while those who have a fixed mindset are more likely just, as you said, to use a small number of exams as high-stakes assessments. Is it possible that some of the effect is just from better teaching practices and using more evidence-based teaching approaches that give students these opportunities for more retrieval practice, more spaced practice, and so forth?

Elizabeth: I think that’s very possible. However, we’ve done some follow-up research where we randomized faculty messages within one course. So it’s the same professor, it’s the same structure, some students are getting growth-mindset messages from that instructor, some students are just getting control messages, and we’re seeing really great effects at that level. And so it has to be more than just the way their course is structured. It’s more about the messages that they’re giving students and how to frame mistakes, how to frame ability. We’re providing it at a specific time like I mentioned earlier, and so it’s probably a combination of both. But with this new experimental evidence at the student level, or at the classroom level, we’re seeing that it’s more than just their teaching ability, or the way that their course is structured.

John: What you just described reminds me of a podcast we had done a while back with Angela Bauer at High Point University in Episode 49. In that episode, she talked about trying to reduce some of the achievement gap in their introductory biology classes. And they first introduced some active-learning activities, but there was still a non-trivial achievement gap remaining. So they introduced some growth mindset messaging, and that seemed to remove the remaining racial achievement gaps. So that provides a little bit further evidence that growth mindset messaging can play a significant role in helping to reduce these achievement gaps.

Rebecca: One thing that’s really powerful about that idea, though, is that it may not really take a lot to make a change.

Elizabeth: Yeah.

Rebecca: It doesn’t really seem like that much of a financial investment, or even a time investment to make a difference, or at least chip away at the problem a little.

Elizabeth: I think that’s what makes this so appealing to people is because it’s subtle, and it’s also something that can be done pretty easily. So you’re not changing the curriculum, you’re not flipping your classroom that requires extra work and time. Faculty are already overloaded with the expectations of what they’re supposed to be doing with teaching in terms of all their other responsibilities. We have implemented mindset messages at a pretty really basic level. So putting messages in a syllabus, putting messages in an email, maybe a couple videos in the class, for instance. And that’s really it. As long as it’s done in a way where it’s at a critical time, it’s more meaningful for students, and it’s done sincerely, then it’s not a whole lot of extra work.

John: From a faculty member’s side though, for those who may have a fixed mindset who believe that students’ ability is fixed, a conversion to a growth mindset may very well, and that’s consistent with your results, change the way in which they structure their courses. Because if you believe that students can learn by making mistakes and practice, you’re probably going to redesign your courses to build more of that in and that’s, again, very consistent with what you found. And it will be a bit more work typically for instructors unless they can do it in a way in which there’s some degree of automation.

Elizabeth: Yeah, I think there’s a lot of discussion around rigor, and if you have a growth mindset that it means you’re too soft, and you’re not having a difficult class. So it brings up all these questions about course difficulty and rigor. And I think our perspective and some of the follow-up work that we’ve done in this paper, and in other papers, that the perspective of what is difficult or what is rigorous really depends on who you ask. So if you ask a faculty member how difficult their class is, or what it means to have a difficult class, that’s going to vary quite dramatically from the students’ perspective. So what we’ve seen is that students actually find the growth-mindset professors to be a little bit more difficult or challenging than the fixed-mindset professors, and it’s for that very reason that they have more work to do in the class. [LAUGHTER] They have to make improvements, and they have to redo assignments, and the workload is maybe even a little bit higher, versus a fixed mindset class that might have a midterm or a final, there’s less, quote-unquote, “work” to do in that class. And so it really is in the eye of the beholder what class is difficult and what that means in terms of student achievement.

Rebecca: There’s a difference in regular accountability…

Elizabeth: Yeah. [LAUGHTER]

Rebecca: …and therefore feels like a lot more work when you’re being held accountable on a regular basis. [LAUGHTER]

Elizabeth: Yeah, I’ve done some follow-up work, because a lot of people will also think that it just comes down to being nice. If you’re nice and warm and friendly, then people are going to do well in your class, and if you’re not, then people will not do well in your class. And so I’ve done an experiment where we have manipulated that. There’s a laboratory study that we’ve done, where we manipulate whether the message is a fixed-mindset or a growth-mindset message. And then we manipulate whether that message is delivered in a way that’s warm and friendly and positive, versus cold and unfriendly. So this results in four different ways of messaging. The best is always going to be sort of this warm, friendly growth mindset, and the worst is always going to be the cold, unfriendly fixed-mindset. But what’s really interesting is what happens to the two in the middle. If you have the growth mindset but you’re cold and unapproachable, versus a fixed mindset and warm and approachable, How do those two shake out? And what we’re finding is that, very consistently, it’s the mindset message that matters more than the warmth or approachability. So they do interact, of course, it’s good to be warm and friendly. But the message itself also matters. So we can tease these things apart. They are different things, and they have different effects.

John: Since we’ve got you here, we were looking through some of your other research, and you’ve done a number of studies that have looked at the impact of utility-value interventions. Could you talk about what those are, and what you’ve found involving those?

Elizabeth: Yeah. So a lot of the research I’ve done on utility-value interventions was conducted in collaboration with Dr. Judy Harackiewicz at Wisconsin, Madison. And so, utility-value intervention is an intervention that’s directed at students. The way that we implement it is we have students write a short essay. They pick a topic that they’ve covered in class, and they write about how that topic is useful or relevant to them personally. And the way that we’ve done this, we’ve had them do it a couple different times throughout the semester. So they’re reflecting on what they’re learning and they’re connecting it to their own personal goals, their life, and it makes the material that they’re learning more relevant to them. And what we’ve found is that this intervention is particularly beneficial for underrepresented racial minority students who are also the first in their family to go to college. So this group is doubly disadvantaged, they’re disadvantaged due to race, they’re also disadvantaged due to social class. These students have the largest achievement gap in most STEM courses, and they really connect with this intervention. What we found in this research, these students have a particular motivation for going into science. They want to give back to their communities, they want to help their friends and family out after they’re done with college. They have specific goals that may not be met in science courses, or may not have this direct connection to what they’re learning in science. And so, by providing them with this opportunity to reflect on that and connect the material to those goals, we’re seeing that they make marked improvement in terms of their grades in the class. Whenever you do an intervention where you have students write something, and you get really rich data because you can look at what they’re writing about. So we analyzed over 1,000 different essays that these students wrote and we said, “What are they writing about?” We ran them through the linguistic analysis, and they’re really connecting it to those goals. These students are more engaged in the assignment, they’re writing longer essays, they’re more specific in their writing. And that then contributes to learning more in the class, which results in higher grades.

Rebecca: Sounds like, again, a very easy intervention to make or to build into classes. I know that I’ve been doing more of that in the classes that I’m teaching and seeing really good results and having really great conversations as a result with students as well.

Elizabeth: Yeah, I think there’s a difference between simplicity and ease of this type of intervention, because it does require the students to put in some effort, they have to write an essay. It also requires some grading on the part of the instructor if that’s not already in your class. So it is a little bit extra, quote-unquote, “work”. But I think it can be done in automated ways to benefit students.

John: And in these studies, you’ve also looked at the effect of student-identified value versus when it’s communicated from the instructor instead. Could you talk a little bit about the relative impact of directly-communicated utility-value interventions from the instructor or those that come from the students?

Elizabeth: Yeah. So from a really practical perspective, we wanted to know, “Do students have to write the essay? [LAUGHTER] Do we have to grade all these essays? Or can we just stand up in front of the students and sort of give them this information, kind of summarize how what they’re learning is relevant to a number of domains?” Because that would ultimately, in a practical sense, take less time. So we did an experiment where we manipulated this, and what we found is that students benefit the most when they get both. So they have a little bit of scaffolding from the faculty member, where they’re given some ideas of how the information might be relevant or useful to them. And then they write about it in their own words and get really into specifics. So a professor can stand up there and say a bunch of ways that it might be relevant, but every student is unique, every student has different goals, every student has different interests. And so, it really needs to be personalized to them. And the process of putting it in your own words and reflecting on it is also useful, right? That’s part of the learning process. You get into the specifics of it and write about it. So, we ended up concluding that it’s both. There’s some scaffolding involved from the faculty member, but then the students really need to generate something for themselves too.

John: I think you also looked at this in terms of the differential effect in two-year and four-year institutions, and you found somewhat different results between a community college and a four-year institution. Could you talk a little bit about that difference?

Elizabeth: Yeah. So a lot of this research has been done, or the data have been collected from four-year institutions, particularly four-year research-intensive institutions. And so, I wanted to see how this could translate to other types of colleges, particularly in the two-year context, because a lot of first-generation students go to two-year colleges. That’s a gateway to a lot of different career paths. And so what we did is we went around to a whole bunch of different two-year colleges, we connected with the instructors there, we tried to tailor it for their students and their context. But ultimately, what we found is that there needed to be a lot more scaffolding in terms of the writing process. So because the intervention was done with students at four-year colleges, we kind of made assumptions about how ready students were to think about utility and think about relevance. And a writing intervention in those contexts just wasn’t appropriate. So we didn’t find the same findings, we actually found that the control essays were more beneficial than the utility-value condition, in these interventions that control is summarizing course material. And that was actually really beneficial for students in this context because they weren’t already doing that in these courses. Whereas in the four-year college, that was sort of the status quo, and they were able to take that next step to make that course material relevant to them. So in working with their instructors, what we concluded is that you can do this intervention in a different way. It doesn’t necessarily have to be writing. It could be done in small group discussions, it can be done in presentations, it can be done in a lot of different formats that might not present a barrier of writing ability or writing practice. A lot of students in two-year colleges take concurrent writing courses in addition to their science courses. And so removing that barrier of writing, I think, would have been necessary in that context.

Rebecca: It’s a nice helpful reminder, I think, for faculty to be thinking about ways to have reflective practice that doesn’t always involve a lot of reading too. [LAUGHTER] Like, if we’re doing presentations, or if we’re reflecting in a video, or reflecting in conversation. These are all other places that provide some variety, too, so that we’re not always grading the same things or having to intake the same kinds of information. That can also be overwhelming to faculty too. So mixing it up is helpful I think. [LAUGHTER]

Elizabeth: Yeah, I think it also speaks to, you know, one-size-fits-all interventions just are not appropriate. So we publish these papers and we say, “Wait, we found these really amazing findings.” But that’s in one context, maybe with one instructor, maybe at one institution, and every student body is different, every class is different. And so, you really need to figure out the needs of your students and meet them where they are, and also take a step back and look at the purpose of the intervention. Maybe it can be implemented differently. Maybe if you take the philosophy of it and customize it for your context, that’s going to be the more appropriate approach.

John: We always end with the question, “What’s next?”

Elizabeth: Well, right now I’m still working on mindset messages. We just got word that we might potentially be getting a nice grant to look at this. And so what we’re going to be looking at is growth-mindset interventions directed at students crossed with growth mindset messages from faculty. The idea is that when you try to get students to believe in a growth mindset, it’s going to be most effective when the environment is supportive of that message so that it’s supported by the faculty member in that class. So we’re going to be looking at that over the next few years in a bunch of different contexts, in a bunch of different institutions.

Rebecca: Sounds like more beneficial, useful, and exciting information.

John: And again, as Rebecca had said earlier, these are really relatively simple and easy-to-use interventions that I think could be much more widely adopted. Well, thank you.

Elizabeth: Yes, thank you for having me. It was great to talk with you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]