262. What Teaching Looks Like

Video recordings of faculty teaching classes have long been used for professional development. In this episode, we examine Martin Springborg and Cassandra Volpe Horii join us to discuss how still photography may also be used for this purpose. Martin and Cassandra are the co-authors of What Teaching Looks Like: Higher Education through Photographs. Martin is the Interim Dean of Liberal Arts and STEM at Dakota County Technical College. Cassandra is the Associate Vice Provost for Education and Director of the Center for Teaching and Learning at Stanford University.

Show Notes

Transcript

John: Video recordings of faculty teaching classes have long been used for professional development. In this episode, we examine how still photography may also be used for this purpose.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Martin Springborg and Cassandra Volpe Horii, co-authors of What Teaching Looks Like: Higher Education through Photographs. Martin is the Interim Dean of Liberal Arts and STEM at Dakota County Technical College. Cassandra is the Associate Vice Provost for Education and Director of the Center for Teaching and Learning at Stanford University. Welcome, Cassandra, and welcome back, Martin.

Cassandra: Thank you.

Martin: Thanks for having us.

John: Today’s teas are: …are either of you drinking tea?

Martin: Oh, yeah. In preparation for this.

Rebecca: Awesome.

Martin: So normally, I drink coffee all day. Today I’m drinking one of my favorite drinks, a London fog, which is Earl Grey tea made into a latte.

Cassandra: Very nice. That sounds great Martin. And I have an iced Jasmine green tea, given the summer nature of what we’re doing and being out in California, but I had my coffee earlier.

Rebecca: [LAUGHTER] Well, I appreciate the real effort to have good tea for the day. I have a jasmine black tea today.

Cassandra: Another Jasmine drinker.

Rebecca: A nice summer flavor.

John: And I have spring cherry green tea.

Rebecca: We’ve invited you here today to discuss What Teaching Looks Like: Higher Education Through Photographs. Can you talk a little bit about how this project got started?

Martin: I started this project when I was teaching photography courses, and specifically an Intro to Photography course at Inver Hills Community College in Minnesota. So I had been teaching, I don’t know, maybe three or four years, going into this a little bit in the book, but I was doing nothing but teaching. So I was really wanting to do some of my own work again, and I decided to make this project that both I can work on and my students can work on at the same time. We were learning about documentary photography at the time. We’re talking about photographers, like Robert Frank, other photographers in this area. And just really getting to that subject, I encouraged them to make photographs of their lives as students. And I told them, I would make photographs of my life as a faculty member. So they could just see what that looks like. And I could learn about them. It blossomed from there. So it was a great project in class, and I loved it so much that I wanted to keep doing it outside of class, and even when I moved out of teaching and into educational development, and away from the college, I started doing this project, or kept doing this work, and actually named it something and said I was going to do this thing on my own and made some photographs of Minnesota State Colleges and Universities, then that’s where our system was called. I think I presented the work at the POD conference. Cassandra was there, and she liked the work, and she invited me out to CalTech, and I’ll let her take it from there.

Cassandra: Yeah, I was blown away by these photographs when I first saw them. It was around the time that Martin and I were both interacting through the professional organization, the POD Network, and working on conference organizing. So we had some committee work together, and they just stopped me in my tracks. I’d never seen anything like them. I hadn’t ever seen represented what happens in classrooms in this visual, but also visceral, way that got to the heart of what mattered, what I loved about being in a university environment. And at the time, I was starting up a new Center for Teaching and Learning, really working on communicating what teaching was all about, how to engage with teaching. And so Martin’s visit was, I think, the first outside of Minnesota. We got him into classrooms of a great variety of different formats, lecture formats, but also labs, and discussions, and conversations between faculty and students and teaching assistant meetings behind the scenes, and that work ended up being so impactful and important in the change process for our institution. It’s something that we started talking about and building on. And then I think, as Martin went to other institutions, became a part of the conversation. We also had the thought at some point, “Hmm, what if we talked with the faculty whose classrooms were photographed and started to really understand the impact on them.” So from that, we developed a protocol for consulting about teaching with faculty using photographs. And again, it just grew from there, and I think Martin could tell us a little bit about the inspiration for, actually, the book.

Martin: Yeah. So from there, we were talking with so many faculty after making photographs in their classrooms, and really just, in a lot of cases, shadowing them throughout their entire day, in other roles they were in. Getting faculty to reflect on their teaching practice, and other things about teaching, related to teaching, it was just really inspiring, and we thought, “well, this can’t be just kept between us, just can’t be this secret.” So then we started to dig into what else had been done, like if anything else had been done in this area. What could we learn from that? And how could we lend what we had done to whatever scholarship existed out there on this topic? We saw scant offerings out there on this topic. Some people who were doing video consultations, which is a very common way of doing teaching consultations with faculty… set up a video camera in a classroom, let it roll and then have the faculty member watch it and then talk about it. Nobody was really doing the still photograph approach to that. So we put together a book proposal. And really, the book proposal, I think originally was nine chapters, we narrowed it down in scope a little bit. But basically 15 years worth of documentary photography, institutions across the United States under the broad title of “The Teaching and Learning Project.” We billed it as the most comprehensive photographic exploration to date of contemporary post-secondary education in the US, which it is. And then the book proposal also included in those chapters, insights about the state of teaching and learning, challenges and solutions, and how and why to integrate photographs in an educational change and improvement work at organizations, institutions at that college level. And now, we’re hearing back from a lot of people who are diving into the book, and they’re just saying that it’s timeless. And there’s something we really didn’t expect to see. We got some feedback early on in the book proposal process that, well, now that we’re in COVID, this is just going to play as a time capsule of what teaching was. But that’s not what it is at all. And we’ll get more into that I’m sure later as we dive into how it’s being received now. But yeah, that was really the crux of the proposal.

John: What Teaching Looks Like is part of the Elon University Center for Engaged Learning Open Access series. Why was open access important for you as authors for this project?

Cassandra: Yeah, thanks for that question. So as you’re probably getting a sense, and you’ve taken a look, I’m sure, this is a very different kind of book about higher education and for higher education. Not only does it contain photographs, but it’s probably half photographs, it’s nearly 200, and those are really integrated with the text so that they sit in this co-equal fashion, the purpose is also somewhat different. So given these unique factors, we wanted it to be as fully accessible as possible in format and also in terms of cost. So we found that the editorial team at the Elon University Center for Engaged Learning, including managing editor Jennie Goforth, and a series co-editors, Jesse Moore and Peter Felton also had that clear vision and dedication. They were very excited to be able to make the supplementary resources readily available, and to explore the capacity, I would say, and the possibility of this format. Just for one example, we worked very closely with the editorial team on exactly how and what kind of descriptive text would be provided for these nearly 200 photographs. So that descriptive text is embedded right there in the electronic format, in a way that works with screen readers, and it’s also potentially enriching and available to all readers. So the other piece is that we really believed in the potential and the possible impact of the book and helped us zoom out for a moment. And, you know, through your conversations with authors that there are so many more excellent high quality books available and emerging about how to teach in higher education, how to teach in different disciplines, through different methods, and we think those are incredibly important. And now we’re seeing more books, volumes, articles coming out about how change works in higher education, how to structure change, strategies for systemic change, also incredibly important, and Martin and I are working on those things too. The thing that What Teaching Looks Like does that I think is different is it gets at why to change, why teaching matters, why to improve, and provides some new tools in our collective repertoire that engage people’s whole selves in that work. So when we talk about culture change, which is so important to sustainable and systemic change, there has to be that drawing in and sustainable interest in participating so that it’s not purely top down, it’s not a forced experience. The other thing we know is that reflection is so important, pausing to really deeply think about why we teach in particular ways and how to go about change, also incredibly important. And what we found was that, like Martin said, we were missing this whole avenue of engaging with people, and we need everything we’ve got to really tackle the challenges in higher education. So kind of circling back around to why open access. These factors are not necessarily a well-tested formula for publishing in higher education. But again, we think it’s really vital. And as Martin mentioned, the early reception to the book is indicating that it’s landing, so we’re getting messages back in email that you People are saying, like “I’m blown away. This is reminding me what I love about teaching. It’s making me think in new ways,” …and a real sense of excitement about the things that they want to do and the conversations that they want to have with their colleagues.

Rebecca: As an artist and a designer, I was really fascinated by this project since Martin first mentioned it to us, a while back, in part because we don’t typically think of SOTL as something that is visual, or that is documentary in nature. Can you talk a little bit about the methodology and what it reveals that some of these other methods don’t? You’ve started hinting at some of these things, but I’d love to hear more about “why photography?”

Martin: First of all, since high school, it’s the way I’ve communicated. So I’m used to communicating that way. And I think you touched on something.SOTL is typically text based. There’s a lot of great stuff there. But you’d have to dedicate yourself to sitting down with it for quite a long time. And if we’re asking faculty, who was really our main audience for this work, to learn from something, photography was just the first place I went, as far as just being a SOTL project, because still photographs instantly land with you. If you look at something, you have an opinion about it, whether you like it or don’t like it, whatever you see in there, immediately, it hits you at a very gut level, the longer you sit with something, the more you see the intricacies of it and more maybe the message that was intended by the author or the photographer, but because it was familiar to me as a way of communicating, that’s really why the methodology, and that’s why I see it as different from a lot of SOTL work is because it takes so little time comparatively, to get into it. You can use photographs to teach faculty something in far less time.

John: And imagery can be really powerful. When we think back to many historical events, key photographs from those events often have a much longer impact than any text that was written up describing those events. So I think you’re onto something quite interesting here. But could you talk a little bit about the structure of the book, and what readers can expect to find in each chapter?

Cassandra: Yeah, I’d be happy to take you through the outline of the book a little bit and preview what you can expect. So in the introduction, we talk a little bit more about the origins of the project itself and some of that surrounding work, which we alluded to a bit here. We also give readers a chance to anticipate how they might interact with the book and think about some strategies for observing the photographs, for taking that pause to really look closely. We continue to do that throughout, of course, but there’s this little primer in the beginning for just interacting with this type of a book, several of the chapters take up place, or a kind of experience in higher education, because we started with really the thematic commonalities that were coming through in these really 10s of 1000s of images that Martin had from multiple different campuses across the United States. Those often pointed to these thorny, challenging questions about what’s current, what’s difficult, what’s interesting about being in higher education today. So in classroom interactions, one of the chapters, it starts out with this quote from one of our participants, “if I were lying on my deathbed, I would want to look at these pictures to know that I did some good teaching.” When we heard that, we just got goosebumps, at least I did, and it speaks to this visceral power that we’ve been talking about. That chapter also looks at what the absence of photographs to date says about the value we place on teaching, what kinds of things we learn when we do have photographs. It’s interwoven with some of the theory and critical perspectives on what photographs are and how they function. We also look at, in another chapter, student perspectives, and really think about the student experience today, what kinds of emotions and forms of engagement, collectively, we’re willing to showcase and what we might be hiding from students that could get in the way of the deeper learning that we’re trying to get at, as well as we know that roles for educators are shifting as we move more toward active and engaged forms of collaborative learning. We also think in another chapter about our interactions with technology and spaces, that interplay that’s also speaking to the importance of those resources. They set the stage and the kind of agency that we might still have and the power relationships that we do have the capacity to change in those environments, and even looking at the nuances of partnerships beyond campus, of community engaged learning. There’s a set of chapters too that take on a little bit more of a philosophical slant. So we look at the productive chaos and the messy nature of education… a sort of deeper exploration of order and disorder, and really a critical admission and questioning of perhaps, the sort of secret aesthetic that we might come to in our experience as educators. Many of us might think that there’s something beautiful about learning, but it’s not something that we say, and we need to say it, but we also need to question it. And then there’s also kind of a real look at the behind the scenes, hidden work, the hidden labor behind education that we almost never show: the work of contingent faculty, adjunct faculty, tenure-track faculty, alt-AC staff, administrators. And we really see the liminality of some of those positions showing up in the images themselves, but also the capacity for that to connect across roles and generate empathy. And then the arc of the book ends with thinking about photographs and change agents, and how they can play a role in campus communities, making intentional, institutional, and educational change through the ways we communicate, how we bring people together through exhibits of photographs, through having this representation in our shared spaces, and then how photographs can have an impact on our teaching practice through the conversations that we might have about them. So every chapter ends with some open-ended questions for reflection, and those are expanded on in the website resources.

Rebecca: I really appreciated seeing the collection of photos and what you were saying about the absence of such photos existing or that documentation existing really resonated with me. I’m working on our graduate student orientation in our new learning management system, which is much more visual, and I’m trying to collect images and represent some ideas, and what I was quickly realizing, at the same time I was looking at your book… this was happening at the same time… was, “Wow, we’re not really great, just generally, taking photos of what learning really looks like or what being a student looks like, or what being a teacher looks like, what being in grad school looks like, because a lot of the things are very staged or focus on extracurricular spaces or they’re just very staged…” it was thinking about other opportunities where I’ve gotten to see something which I appreciate being able to see something in action, but often those seeing of the things in action are staged.

Martin: I’ve given talks to faculty at institutions when I’ve gone to to make photographs. And that’s one of the things that is often in those talks is wrestling with how we project ourselves to potential students and their families, the balance between marketing and what they would love to do, and the reality of what students see when they get to college. They’re often not the same thing. So we project a lot of images that don’t represent ourselves in truth. And I think that’s potentially why students are shocked often when they get to college. And they realize that, “oh, this is a wild mess,” in a lot of aspects. “This is crazy. I don’t know if I can take it.” What I’m trying to say is, if we do have more photographs available to students, sadly, if we’re able to have discussions around those truthful photographs… I won’t say truth… but around those honest photographs, then I think we circumvent a lot of that, we take some of the shock away from what it is to be in college, what it is to be a student, what it is to be a faculty member. If we showed photographs like this to our graduate students more often, they would understand what’s ahead of them when they teach their own classes. They get that in the teaching assistant roles, of course, but there’s more in those photographs to learn in that respect.

Rebecca: Yeah, the authenticity of the collection is really impactful. I really also appreciated all the companion resources, and would love for you to talk through some of those. There was multiple things that you’ve included, not only within the text itself, but also as this companion material on the website. And there were a couple that I was particularly interested in that I hope you might walk us through. One of them is the close reading and observation activity. Can you share a little bit about that?

Cassandra: Yes. And in fact, if you’d like we can try that out a little bit right here. Are you game for that, Rebecca and John?

Rebecca: We can try. We can try anything.

Cassandra: Alright. So the companion resources, as you alluded to, includes a set of close observation, close reading exercises, which excerpt, one photograph or sets of photographs around several themes, and then offer some prompts for reflection, conversation, discussion. We imagine that these… and they are actually…. being used in a reading group and book groups. I have given workshops on my own campus and Martin and I have done similar things in conferences where we take this similar method and really work with a group. It’s a great warmup, gets people very engaged, gets them thinking, also a wonderful activity in the context of reflecting on teaching. So lots of different uses, but I’m gonna have John and Rebecca, and Martin, if you like, open the resource called close reading and observation exercises, just to the first photograph that you find there. So it’s on the second page. And it’s called Introduction, close reading exercise. Is everybody there?

John: And we will share a link to all the resources, especially this one, in the show notes.

Cassandra: Fantastic. So let’s just start out by engaging, looking at this first photograph. And I’m going to ask John and Rebecca to share what you notice, in this image… start to describe it. Of course, we’re on audio. So your description will be helpful for listeners, and it will tell us something and yourself something about what you think is important and what stands out. What do you see?

Rebecca: I just want to verify that we’re looking at the figure 2.01.

Cassandra: Yes, 2.01.

Rebecca: So what I see is two groups of students, and each group is organized around a computer monitor. In the group that’s closest to us, I see a diverse group of students. And there’s one student who is gesturing with his paper at the screen, two students leaning in to see this screen, and one student hanging back a little bit… looking, peeking at what’s on the screen, but not quite leaning in like the other two are.

Cassandra: Awesome. Thank you. So there’s no wrong answers here. All of those are things that are present in the photograph. John, is there anything you would add, just at first glance, of what stands out to you?

John: Well, the students all seem focused on the same material, they all seem to be actively engaged in the activity, which is not something we always see in classrooms.

Cassandra: And as you think about, look at this photograph a little bit more, is there anything in it that maybe seems typical or atypical?

Rebecca: I think the level of focus is not always typical, as John mentioned, but it’s also, like a super win [LAUGHTER] when that level of focus is there. So I’m feeling the winning happening. It gives a teacher who is probably walking around the classroom, and you’re feeling like, yeah, the students are into this thing. So I can’t see that. But I’m feeling that.

Cassandra: Yeah, well, and that emotion is coming through in the details that you already have observed. So Rebecca, you pointed out some of the expressions and the body language that you’re noticing in this photograph, some of those signs that you see about what engagement might look like, what happens when students are getting engaged, that sort of leaning in, and the gesturing that’s happening. So there’s this action that you’re noticing as well. One more little question is this, you might not be able to tell from the photograph, the person closest to the lens, whose back is toward us, and who has the piece of paper gesturing toward the screen is actually the professor, the teacher in this setting. So any thoughts about knowing that about this image now, what might be going on and what you notice about the roles?

Rebecca: To me, it seems like the faculty member is checking in on something that’s happening, and now they’re having a conversation and maybe some explanation of where there was a misstep. And really the leaning in of like, “Wait, I want to know the answer. I want to understand this better.”

Cassandra: Yeah, fantastic. So I don’t want to go on too long. But I think, hopefully, that demonstrates that just with that moment of looking and some prompts and questions, you start to really get deeply into the photograph, deeply into the experience. And if we were in a course design institute, we might use this prompt and this image, to start to open up a conversation about how we want to structure in-class work, once we have some learning outcomes defined, the big goals of the course. And think about different ways we want to get students engaged and what that’s going to be like for the instructor. If we’re thinking about active learning, we might use this as a starting point to really reconsider the role of the instructor, the moves that they need to make, how to check in with students, we also might be thinking about student groups and how they’re interacting. So there’s lots of ways we can go and it’s that sort of open-ended, reflective quality that is really exciting, and I think fun to engage in and can open up some new possibilities.

John: And it’s especially nice to see all these resources provided with an open access piece of work, because that often doesn’t happen. These are really useful and powerful supplements that make these tools much more usable for professional development.

Martin: Another thing that I want to mention, in reference to this question about additional resources is the guide to photo-based consultations that Elon has on their site. So this is just really us offering our complete template to conducting a photo-based teaching consultation. It’s exactly, with very little modification, what Cassandra and I used every time we sat down with a faculty member that we photographed and guided them through reflection on their teaching practice using the photographs as a reference point, and I think it’s a very powerful way of conducting a teaching consultation, it was often an emotional experience for the faculty that we talked to, and I don’t even remember how many of these we did, we did quite a few at several institutions. They’re all very valuable experiences.

Cassandra: There’s a standalone published article about the teaching consultation framework and some of the findings about what we coded as incidents of reflection and different kinds of reflection that came through in these photographs. And then material from that same study is also incorporated into the book. So readers will see it again, less as the formal research study, and more as a compliment and an accompaniment to the images around these different themes.

Rebecca: We’ve talked a bit about the photographs that you’ve taken and the discussions around the photographs. Can you talk a little bit about the process of actually making the photograph and what the interaction was like with a faculty member to set up that opportunity? And then to follow them around all day? [LAUGHTER] And that was like, and how do you keep that authentic?

Martin: I’ll just start by saying it’s not easy, and it especially wasn’t easy in the very beginning. The first people I photographed were my colleagues, because they were colleagues and also friends, they trusted me. They’re like, “yes, you made this work in your class of yourself, for your students. But how are you going to do that of me in my class?” So it took some getting used to in the very beginning, and then we have like a two pager guide to making photographs in the resources on Elon site as well. But the biggest thing to remember is that things were more than likely be a little weird for about five minutes, you just have to keep making a lot of pictures, and only after you make a lot of pictures of the first few minutes that you’re there that people just tune you out, it does happen. You just have to trust the process, it’s just, you see all the photographs in the book. I had been in those classes with that faculty member trailing that President for some time, and they acted as though I wasn’t even there. And then the photographs, you can see that the people are acting naturally, as though nobody else is in the room with them. That’s my biggest piece of advice. For those that are wanting to do this on their own, I think that the biggest thing to keep in mind is that those moments will come. And you have to remember the ultimate goal of why you’re there and what you’re doing.

Cassandra: One question that we often receive is about permissions for the photographs, which is very important to protect the privacy and let people course consent and elect in to being represented in photographs. So everything that’s in the book, and in Martin’s body of work, every single person has consented actively and signed a photographic release. And when you’re working in your own institution, it’s a good idea to consult with your communications colleagues, potentially General Counsel, just to make sure that you’re following your own institution’s guidelines. Sometimes there’s a blanket photographic release that students elect into or out of that you can access. And in this case, because Martin was also exhibiting and presenting the work outside of the institution, that separate release was very important.

Martin: It’s important to note as well, that the release form gives equal use rights to both myself and the institution. So we have a shared ownership of the images, when I’m finished photographing at a location. And I just give them a complete set of the photographs that I made, so that they can use them for whatever purposes they need to use them for. I’ve almost always worked just with Center for Teaching and Learning staff. They’re going to use those photographs to really put teaching and learning out there visually as a priority at their institutions, and many of them have done that.

John: You visited many campuses and lots of classrooms. What were some observations that stood out in terms of really effective practices or effective activities that were occurring in these classes?

Martin: That’s a tricky word, “effective,” I think. So I’m going to dodge that question, I think, a little bit and talk about the different things I saw that on face value wouldn’t seem like they would work in terms of teaching students things, but ultimately did work to teach students things. So one part of the book that Cassandra mentioned earlier, that messy nature of teaching and learning gets into a situation I was in at one of the institutions where it was this kind of like a giant office hour or recitation session for students in an enormous class with three or four hundred students. It was right before an exam. There was a lot of nervousness in that room. Students were crammed in there together and they were all just sort of tossing notes back and forth… literally through the air, in some cases… handing laptops back and forth from table to table. But in that mess, were also a lot of TAs and the professor of the class, just making round after round after round, consulting with students. And that, if you were to just walk by, would be very loud, as it was loud and chaotic. And you would think “what’s happening here? What good could possibly come from this?” But if you get into the nuances of those photographs, those still photographs, you’ll see that sort of emotion and caring that’s in those interactions between the TAs and the faculty member and those students. And you’ll see on student faces, those moments where they’re like getting it, those “aha” moments. If you teach, you know the face, and those faces, or in those still images, I’m gonna dodge the most effective methods and go to those kinds of moments where I discovered something that to the outside wouldn’t appear to be working, but actually was working very well.

Rebecca: Were there any other really prominent moments, Martin, that really stuck with you?

Martin: There are so many. One situation I was in, I’ll always remember it. So I photographed a large lecture class, it was at night. So there were some students at the back of the room, and the hockey game was on their laptops. They weren’t paying attention to the lecture, it was nine o’clock at night. And then there were other students very engaged in that class, what I wanted to do was capture everything that was happening. When I make photographs of a complete class, which is always what I do, I never want to go into a class and photograph for 15 minutes and exit, I want to be there at the beginning before class starts, and then photograph even after it ends, and students coming up and talking or asking questions. Because that’s the whole thing. So I make probably hundreds, easily, of photographs. And then I whinnow that down to about 60 or so. And then I give those photographs to the faculty member to reflect on for teaching consultation moments that we have, because there are bad photographs, you’re gonna make a lot of bad photographs. But I try, in those 60 or so, to give them a summary of what happened from everything that was happening in that class. Well, this one faculty member just didn’t believe that that was what was happening. He just didn’t believe it, and wanted to see every single photograph and the timestamp on the photographs to know exactly that I got the whole class. And he wanted to see them not in black and white, he wanted the color just because he knew that they existed. I take color photographs, and I make them black and white. So that stuck with me and the reaction after I said “fine, I don’t mind giving you all these, here they are.” But the reaction after that was like, if you can summarize the email that would summarize it as like aha moment where it’s like, that really did happen. All that stuff was going on. I’ll never forget that. I don’t know why. But that’s one moment, one opportunity that stands out to me amongst all the others.

Cassandra: On the flip side, if I can add, there have also been these really striking conversations when instructors were having the chance to observe maybe the student in the back corner of the room that they maybe don’t engage with directly as much or who is quieter, and seeing them get really excited when the instructor is somewhere else in the room. So there was also a lot of seeing things for the first time, seeing students get excited, and recognizing the real difficulties that students are sometimes facing. There’s one image that I don’t think is in the book, but I will always remember seeing this and talking with the faculty member about it, where a student was in a class, computers were being used. There’s lots of stuff in the room. But this one student had laid very carefully on the table next to the computer a white food service uniform that was clearly needing to stay clean and crisp and pressed probably for the job that the student was going to go to after class. And that student was also engaged in the class, but clearly juggling a tremendous amount of life and work. And that’s all there in that image for that faculty member to reflect on and understand.

Martin: We could go on for a really long time. I want to mention two other photographs that stood out to me. So there’s one of a faculty member conducting office hours. And in the office, she’s really intently working with a student. And then you can see the student is really struggling to get it, like physically, the hands on the head. And then outside down the hall, you can see a line of students sitting on the floor waiting for their opportunity in the office hour. I think to me that just sums up what office hours should be and what they are. And it’s a thing that people don’t understand. If you’re not faculty you’ve never taught before, you’re outside of higher education, you don’t understand that that’s a part of faculty work. You can see that same struggle lined up in the hall many times more for that faculty member. And then one more photograph that I want to mention, because it’s funny, and both Cassandra and I laughed hysterically about it. So I photographed a large exam happening. This is one of those exams where it’s proctored by a graduate assistant and it’s timed, it’s very, you got to do everything by the book. This one student in the front row taking this exam is in a panda suit, like a panda costume [LAUGHTER], and it’s just a beautiful moment.

Rebecca: [LAUGHTER] These things do happen. I wanted to pick up on one more thread that you had mentioned at the top of the conversation, which is the power of… and you’ve hinted at some of these examples… the power of these photographs to instigate change. Change, perhaps for that faculty member, and I’m pretty sure I heard both of you imply that it could happen beyond the faculty member too, and I’d love to hear a little bit more about that.

Cassandra: Yeah, I could start this one off. And Martin, please feel free to jump in, of course. Some of the ways in which campuses were, first of all, initiating a project to capture photographs, and then really creatively engaging their communities with the photographs were pretty striking. So one example comes from the University of Michigan, and we’ve discussed this and have some wonderful reflections from colleagues there in the final chapter of the book. Looking at the largest courses on campus, this initiative was really trying to understand what was happening in them, what students were experiencing, what faculty were experiencing. And so that set of photos served as both a kind of baseline for the project, and an interactive tool to engage with those communities about how to change those courses, how to make them better, how to consider what would be most important. In other cases, campuses, and we did this actually on my own campus, created exhibits of photographs in museum spaces, but also in other kinds of spaces in a Center for Teaching and Learning, in an important office on campus. We’re hearing of more places that are also doing similar things now. Some temporary exhibits really brought presidents, and provosts, and deans, and chairs together with faculty teaching the courses, with these large-scale photographs that sparked new kinds of conversations, conversations across disciplines, across administrative and faculty, potentially, sometimes what’s perceived as barriers or misunderstandings. And in some cases, we heard stories about people walking off with the photographs because they wanted them for their own offices for their own.

Martin: They were told they could leave with their photographs at that one event.

Cassandra: That’s good.

Martin: They didn’t steal them.

John: One of the things you mentioned, Martin, was faculty being surprised by what was happening in various corners of the room. And that’s an issue that I think might be really enlightening information for people teaching large classes if they don’t normally walk around their classroom. Because nothing you said there seems surprising to me, I’ve been teaching classes of three or 400 students, and I generally get in often 3 or 4 thousand steps during an hour and 20 minute class because I’m as likely to be teaching from the back of the room or working with small groups of students as I am to be up at the podium. Or at least that was true until very recently. I’m hobbling around a little bit right now, and that actually is a concern I have going into this semester, that I’m going to be a little bit less mobile for the first few weeks. I had a bit of an accident a while back that broke my leg in a few places. So I’m very concerned about not being able to be out there with students for at least part of the semester. But I think it does illustrate the importance of being out there amongst the students. And we’ve often heard people talk about teaching by walking around, and it’s a really effective technique, and having these photos can encourage that in cases where faculty members are skeptical about what you observe in portions of the class.

Martin: Yeah, I don’t want to make it sound like that was the only large lecture situation that I found. I did photograph quite a few large lecture courses where the instructor was up and down the stairs, constantly making the rounds around the room all the time with a Bluetooth headset, rather than being behind the podium. You’re right, you talk about effective teaching methods, and that’s definitely one for those large lecture courses. Not only having the instructor in there, but also having TAs, graduate assistants wandering around constantly, and using the time and the space to conduct group work because you do get around the room.

John: We always end with the question. What’s next?

Martin: One thing that we’ve been able to do since the book has come out is engage more communities. I’m still asking myself what’s happening. What is my life right now? Because soon after the book came out, we gave two book talks to a contingent of educational developers in Asia. It’s this organized two events for people who wanted us to talk about the book with them. And just due to the reach of the book, and what Elon has been able to do, promoting it in that way, it’s so exciting to be engaging more and more communities of folks about the work. Also, we have institutions that are inviting us back, and mounting exhibitions of the work. So one example is St. Louis University, this fall in their museum of art is mounting an exhibition of the work that I made there. I’ll be going back there to make more photographs this fall, as well as Brown University. And I guess the third thing I want to mention for next things is we’re just hoping to build on models from the book, and conducting more educational development and teaching related professional development.

Cassandra: I’ll just add, we’re really excited to observe how communities pick this up and run with it, we’ve talked a little bit about those supplementary resources. So it’s really an approach that campuses can adopt, adapt, and run with. So for example, in one of the recent discussions with the SOTL Asia network, one faculty member was very excited to start to work with students on them documenting and sharing their own experiences as a way for them to reflect on their post-secondary experience, and really be able to communicate it in a different way. Others were immediately thinking about all kinds of contexts that they realized had never been shared about their own learning contexts, their own classes, sort of specific forms of special kinds of classes or environments that they realized were really important and should be shown and should be captured, those kind of hallmarks of the institution or the program or the community. So we’re finding that often just this idea of communicating with images brings to mind the images that haven’t yet been made and the engagements that haven’t yet happened about those representations, those forms of teaching and learning. And we’re hoping to have more of those conversations and to engage with more folks around the work that they’d like to do.

Rebecca: Well thank you so much for sharing your work. It’s really interesting, exciting, and really something we haven’t seen before, and so we’re looking forward to sharing it and spreading the word.

Martin: And on that note, I’ll say one more thing about what’s next. We just talked about this the other day. So now we have this visual baseline of what teaching looks like, and we can refer back to it maybe in 10 years and see how teaching has changed visually.

Cassandra: It’s been wonderful speaking with you. Thanks so much for having us on Tea for Teaching.

Martin: Yes, thank you.

John: Thank you both for joining us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

261. Social Justice Assessments

Traditional methods of assessing student learning favor those students that reside in well-resourced school districts while leaving low-income students at a substantial disadvantage. These grading systems also encourage students to focus on their grades rather than on their learning. In this episode, Judith Littlejohn, Meghanne Freivald, and Katelyn Prager join us to discuss a variety of social justice assessment techniques that can help to create a more equitable environment in which all students can be successful.

Judie is the Director of Online Learning at SUNY Genesee Community College, Meghanne is an Instructional Technology Specialist at Alfred University, and Katelyn is an Assistant Professor in the English Department at the Fashion Institute of Technology.  Judie, Meghan, and Katelyn worked together on a SUNY Faculty Advisory Council on Teaching and Technology committee on social justice assessments.

Show Notes

Transcript

John: Traditional methods of assessing student learning favor those students that reside in well-resourced school districts while leaving low-income students at a substantial disadvantage. These grading systems also encourage students to focus on their grades rather than on their learning. In this episode, we explore a variety of social justice assessment techniques that can create a more equitable environment in which all students can be successful.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Judith Littlejohn, Meghanne Freivald, and Katelyn Prager. Judie is the Director of Online Learning at SUNY Genesee Community College, Meghanne is an Instructional Technology Specialist at Alfred University, and Katelyn is an Assistant Professor in the English Department at the Fashion Institute of Technology. Judie, Meghan, and Katelyn worked together on a SUNY Faculty Advisory Council on Teaching and Technology committee on social justice assessments. Welcome Meghanne and Katelyn and welcome back, Judie.

Meghanne: Thank you.

Katelyn: Thank you.

Judie: Thank you.

John: Today’s teas are:

Judie: …I have Lady Grey.

Rebecca: That’s a good one…

Judie: …In my DTL mug.

John: …a nice Desire to Learn mug.

Meghanne: I have iced green.

Rebecca: And Katelyn, how about you?

Katelyn: Mine’s water right now, if it were the evening, I would have one bag of peppermint and one bag of chamomile together, delicious.

Rebecca: Sounds nice and calming.

Rebecca: I have hot cinnamon spice tea.

John: And I have black raspberry green tea.

Rebecca: We’ve invited you here today to discuss your work on social justice assessment. Perhaps, we can start with a discussion on what you mean by social justice assessment.

Judie: So social justice assessment considers factors such as race, culture, language proficiency, socioeconomic status, and ability while working to dismantle systems of power, bias, and oppression in evaluation of student learning. So various approaches including equitable assessment, labor based grading, and ungrading, as they relate to the purpose, process, wording, and structure of student learning assessments are included. So we’re trying to focus on the learning that our diverse students achieve as it relates to specific learning outcomes just to mitigate the influence of dominant norms on our students’ grades. So we’ve all been working together for the last couple of years on a SUNY task group that was part of the Faculty Advisory Council on Teaching and Technology, which I chair. So we’re a subcommittee of an Innovations in Assessment group, and there’s a couple more of us who couldn’t make it today, but we’ve been a really close-knit group, I think, working together for over two years. And we really enjoyed the project, which resulted in a website with all these artifacts on it that people will be able to access. And we’re hoping down the road that we can continue our work, but we’ll get to that later on in this conversation.

John: And we’ll share a link to the overall website as well as your group-specific component of that in the show notes. So this was partly implied in your response defining social justice assessment, but, what are some of the shortcomings of traditional grading systems in terms of equity?

Meghanne: When we were doing our research on this topic, we encountered many drawbacks of the traditional types of assessments that we all experienced all the way up through school and into college, and I’ll share a few of them. One is that the focus is often on the grade rather than the actual learning process and what the student will actually be able to do, and be able to learn as a result of engaging in the education process. They just focus on the grade, “what’s my grade?” and that sort of misses the point. It creates a system where students are compared to each other rather than having the focus be on individual growth and achievement. It also can put students at an advantage or disadvantage based on characteristics such as race, ethnicity, socioeconomic status, disability status, language proficiency, and lots of other characteristics that students themselves don’t have any control over. We found in our research that traditional assessments tend to favor white, affluent, high-achieving students, and that really isn’t who most of our students are anymore. So we really need to remove barriers and create a way for students to accurately represent the learning that has taken place.

Rebecca: So you hinted to this in your response about traditional grading systems comparing students to one another. So thinking about that, what role should students play in determining how their learning is assessed?

Katelyn: I’ll tackle that one, and I want to answer it with a disclaimer to start because social justice assessment is an umbrella term that has all of these different strategies that are wrapped up in it, and each of those approaches, whether it’s ungrading, or labor-based grading, might have a slightly different response to that question. They all share the same goal, that students should not be systematically disadvantaged by the assessment mechanisms, and that we want to increase student agency in the classroom. We want students to be active participants in their own learning, but the actual question of how students might participate in determining their own assessments might look very different depending on who you’re talking to and what approach they use. Maybe it’s literally helping design the assessment mechanisms, the grading contract, grading rubric, maybe it’s creating flexible assignments that allow students to determine what learning is being assessed, or in the case of ungrading, maybe it’s just deprioritizing the assessment entirely in order to emphasize the individual student’s learning journey through the course. So I guess my answer, tentatively to your question, is yes, students should be participants in determining how they’re learning is assessed by the big how, and why is going to differ.

John: As you noted, there’s a wide continuum of alternative grading policies that can fit under this category of social justice assessment. Some of them are not that much different than traditional practices, and others are quite a bit different. One approach, which is much closer to the traditional grading systems that people are already using is a system of mastery learning. Could you talk a little bit about what mastery learning is and how that could be used in the classroom to provide a bit more equity.

Judie: So mastery learning is, instead of assessing a student or evaluating a student with one assessment, and giving them that grade, the students are able to go back and revisit the content and work again on any material that they didn’t understand or try things over again. So it’s an iterative process, and they should get some sort of formative feedback in between attempts so that they can understand what it is they need to work on and focus on. And this way, it’s more equitable, because the students are able to take the amount of time that they need to work on the assessment, they can access any review materials that they need to establish their foundational knowledge and continue on. And it just really helps the students learn and grow. And I think it’s a great way to establish foundational knowledge. I use it myself in all the history courses that I teach, and I just think it’s a great process. If you think about it, any athlete, that’s what they do. So if you’re learning how to play baseball, how many hours are spent in a batting cage, or like on the pitcher’s mound, how many times do you try again, and again, and then again, until you are able to do it correctly, or do things accurately? So I always liken it to use that sports analogy, because I really think that helps people understand that students’ learning… you have to practice and you can’t tell somebody something once and expect them to integrate it into all the knowledge they already have, and be able to recall it instantly. So I just think it’s a great way to level the playing field of students so that when you move on to the next part of your content, they all have the same foundation, and they’re ready to go forward.

John: And by explaining it to the students that way, in terms of a sports metaphor, it’s something that they can pretty easily connect to, and I think it also would help to promote a growth mindset, which we know is effective in increasing learning as well.

Rebecca: Another assessment strategy one might use is minimal or light grading that falls under this social justice umbrella, and is a bit different than mastery learning. Can you describe what minimal or light grading is?

Meghanne: Yeah, I’ve seen this described in a couple of different ways. This isn’t something that we really included in a lot of our research, so I kind of looked this up just a little while ago and it’s very interesting. And one approach is more on like the whole course level. And there’s another approach that can be taken on an assignment level. So for an entire course, what an instructor may do is that they would assign assignments throughout the semester, but most of them would not be graded, they would be used as like a conversation piece. And they would be discussed and gone over during class, which would then provide opportunities for the students to seek clarification and for the instructor to provide feedback in the moment. So then the assessment then becomes part of the learning process. So then when there are a small number of assessments that are given for a grade, then when the students get to those assessments, they’re not as intimidating. They’re things that they’ve done with their classmates, they’ve done them with their instructors, they’ve done them in class. So I think it’s a very interesting strategy because it removes a lot of the anxiety that students may have around assessment, because it’s just something that they’ve done in their class. Another take on this that I’ve seen is, on an assignment level, something like a paper, something that may require a lot of revision, where when the professor is grading that assessment, they would maybe not take the time to go through and mark all of the grammar and spelling and mechanical errors, but maybe they would look at a section of that, maybe point out some things the students are doing over and over again, but not mark up the entire paper, but just say, “Okay, these are the things you need to pay attention to that are recurring through your paper.” And then as they read and grade that student’s paper, they focus more on the message that the student is trying to convey and the ideas that they’re sharing, rather than the mechanics and the grammar and the spelling.

John: And one common thing I think, to both mastery learning and minimal light grading is that the goal is to provide students with feedback. In some cases that can be automated. Mastery learning systems involve some degree of automation, sometimes by textbook providers, or perhaps adaptive learning systems, or it could be questions that you put together. But if you’re going to provide feedback on writing, it can require a lot more time. And a minimal light grading approach allows faculty to provide feedback on the most important things without taking up as much time to allow faculty to provide feedback on a wider range of topics, which, again, is I think, to some extent in the same sort of spirit.

Rebecca: Light grading can help not intimidate a student with too much feedback. If you see just a paper completely marked up, it might feel like there’s no possibility for moving forward or revising. But emphasizing what’s most important to change, or most important to focus on can help a student prioritize. And this can be really important to someone new to a discipline who might not know what’s most important.

Katelyn: I’m so glad you said that.

Meghanne: There’s an element of trust there as well, because if we point out what a student needs to focus on mechanically or grammar wise in a small part of that paper, then they can be trusted to then use their judgment to go through it and read it more carefully, and then make those edits based on the feedback that they had received. So it is visually much less intimidating. Plus, it might be a motivating factor for some students too that their professor is trusting them to be in charge of that revision.

John: Another type of social justice assessment involves contract grading. Could one of you talk a little bit about how contract grading fits into this category of social justice assessment?

Katelyn: Sure, I think contract grading is one of those terms that’s gaining some broader popularity and recognition. So it’s probably a term that may be pretty familiar to a lot of instructors at this point. So maybe it doesn’t need a lot of explanation. I’ll just say there’s a couple of different models of contract grading. In some cases, the instructor might provide that contract at the start of the term. In other cases, the instructor and students would be able to negotiate that contract collaboratively together at the start of the term so that students have more of that active stake in the contract itself. Generally, the grading contract would lay out certain requirements which students would need to fulfill to receive their desired grade. And that might include requirements related to attending class or conferences, completing low-stakes assignments, completing major assignments, maybe some page- or process-based requirements. But the bottom line is that the contract gives students a clear picture from day one of the work required by the class so students can look at that contract and know exactly how much work they’re going to need to complete from day one, to get the grade that they really want to receive in the course. I think the additional benefit of contract grading for our conversation is that it decouples grades from assessment so students have more space to take risks in their work rather than aiming for correctness. And on the faculty side, faculty can respond to the content and spirit of the students work as opposed to justifying a grade. I think most important, though, because this system privileges students who are investing the time and effort into their learning, all students have the same potential to earn a high grade in the course regardless of their knowledge or ability with the subject matter prior to the start of the course. So to use another sports metaphor, it works to level the playing field on day one for students who may have very different levels of preparedness and experience with the subject matter.

Rebecca: Another strategy that folks might use, which we’ve certainly talked about quite a bit on this podcast at various times is peer assessment. Can you talk a little bit about what that looks like and how that fits into this social justice model?

Judie: So peer assessment, or I tend to call it peer review, helps to build student investment in writing, and helps the students understand the relationship between their writing and their coursework by helping them engage with the writing in a way that encourages more self reflection and works to help them build their critical thinking skills about their own work. And I think it also helps the students learn from one another, because they’re sort of trying to evaluate their peers’ work against the requirements for the course. But then you also look at your own writing in a new perspective, and you learn from what you’re seeing your peers write and from the feedback that you’re receiving from your peers.

John: Might students perhaps take feedback from their fellow classmates a bit more seriously than they do feedback from their instructors.

Judie: A lot of students self-report that they learn more from this peer review activity, because they’re trying to identify and articulate weaknesses that they’re seeing in their peers’ papers, and also in their own. And I think trying to incorporate feedback from both their peers and their instructor into their own work, I think, just helps raise that awareness and any kind of feedback that’s constructive, as they think about it and reiterate it and rewrite their work. It just helps with their critical thinking. And I think just raise awareness of how they write, and maybe they can be more thoughtful about what they’re writing going forward. I think they also, if they question their peers, say “How did you come up with this?I love this idea,” then they can apply some of this, that they’re learning from their peers to their own work, too. So perhaps that’s what you were getting at John, when you asked that question was, they may benefit more from their classmates telling them how they came up with their ideas than from their instructor just dictating what the expectations are.

Rebecca: I would expand the model to include not just writing but also other creative projects and things. It’s certainly a practice that’s pretty common in the arts, for example, to do peer review of student work.

John: And they also get to see what their peers are doing, which can serve as a positive role model. When students see that other people are doing something that they hadn’t considered doing, it could serve as a way of improving their work.

Katelyn: I think a lot of students come into the classroom thinking of their teacher as the sole reader or audience for their creations throughout the course of this semester. So anytime we can expand those audiences and have students thinking rhetorically about who else might be the consumer of their work. I think that that can benefit our students in really important ways.

Rebecca: It also seems like it’s a good opportunity to formulate community around an activity like that.

Katelyn: Absolutely.

John: One of the other areas you address with this group was the topic of labor-based grading, could you talk a little bit about that?

Meghanne: Yeah, labor-based grading removes the focus from the end product assignment and shifts it to the process of creating that piece of work. So students are provided with feedback throughout the process regarding their labor or the work that they put in. And they’re given opportunities to continue working to improve what they’re producing, and to achieve a desired grade based on a contract sometimes, so there is some overlap with contract grading, but not always. There typically aren’t penalties for students who revise and update their work, because that’s part of the learning process. And it really helps students determine what their end grade may be and how much effort they want to put in, because often, they will be given some sort of guideline for what different grades may be achieved based on certain levels of effort, or certain levels of work that are completed. And also there may be opportunities to grade based on completion rather than more of a subjective sort of qualitative grade.

John: So do you mean like using a light grading or minimal grading where you either completed satisfactorily or you haven’t, and as long as you complete a certain number of assignments or activities, you achieve that grade,

Meghanne: That or also if there’s criteria, like a rubric, and they hit all of the criteria, then they receive full credit.

John: Which becomes, actually, I think, a form of specifications grading.

Rebecca: And then one other model that you’ve talked a little bit about already today is ungrading. Can you expand upon that a little bit more?

Katelyn: Yeah, so ungrading works to deprioritize numerical grades or even attempt to eliminate them entirely. So I hope I’m not speaking out of turn when I say, I think that this is the most controversial of the approaches that we have been researching, it tends to get the most pushback from faculty because it is so different from what we have often been taught or trained to do. So instead of focusing on those numerical grades, instructors are encouraged to focus on providing learner feedback that encourages growth. Okay, I have a quote that is from an ungrading expert I’d like to share. This from Sean Michael Morris and he says, quote, “at the foundation of ungrading, lies something that could change school entirely. A suggestion that ranking and evaluation and the concomitant expertise of the ranker or evaluator is entirely an optional way of viewing things.” And I’m going to end the quote there because I think that that important kernel is that ungrading works to dismantle the hierarchy of the classroom and refocus the attention on individual student learning is an approach that requires a lot of trust between student and instructor, and a lot of student buy-in as well. Students have to be invested in the learning that’s going to happen throughout the course itself. And in a completely ungraded classroom, student grades might be based simply on a final student reflection, or even a one-on-one conversation between teacher and student about the grade that the student has earned. But because ungrading really rejects transactional grading systems, the final grade is more of an afterthought than an important outcome of the course, much less important than learning that’s occurred throughout the semester.

Rebecca: So today, we’re recording on August 9, James Lang posted on Twitter about how deep the system of creating actually is that there’s even things like discounts for insurance, for good students, or good grades. And that it’s really challenging to overcome a system that’s so ingrained beyond just our education system, but into many other systems as well. So I think that that, in part, is why there’s such a strong pushback on this particular method.

John: And we’ve always done it that way, at least for the last century or so.

Rebecca: Change is hard.

Katelyn: Yeah, I think that the traditional grading system is really embedded into not only academia but outside of academia as well. And even within a class that takes an ungrading approach, we still face that question at the end of the semester of “Well, what’s the grade going to be in the system?” because we don’t really have the option, at least at most institutions, to say, “No grade, job well done.” At least at my institution, I still have to put in a letter grade for the student. So we can work to reject that system as much as we can. But at the end of the day, we’re still operating within that same structure. And maybe that’s a question of what’s next, right? Like, are we going to see one day a future where more universities embrace this idea of learning for the sake of learning as opposed to learning for the grade? I don’t know.

John: One of the other things you address on the website is how perhaps the use of authentic assessment or UDL types of assessments might improve equity by providing a more equal playing field for students. Could you talk a little bit about how going beyond the traditional term papers and tests might provide a more equitable way of assessing students’ learning.

Judie: I think anytime you use authentic assessment that helps, or generally it allows the students the opportunity to demonstrate their learning in the way that works best for them. The students are writing a term paper, for example, they can write the paper the traditional way, or they can give a presentation or record a presentation, and still provide their citations and so forth at the end. Or they can do something visual, some sort of a PowerPoint or a nice visual display of the topic and again, cite their sources and explain their images to the group so that people understand how they’re meeting the learning outcome. And I feel like that’s just a good way if people are struggling with language, if people are just struggling with writing in general, I think that this levels the playing field, because it gives everybody an opportunity to really show their knowledge and shine and not just pigeonhole themselves into one more paper or one more multiple choice test, if they have test anxiety. Some of our traditional forms of testing or final assessment just set students up to fail. And allowing students to choose to demonstrate their learning in a way that they’re good at sets them up to succeed. And I think that’s what we really want at the end of the day. And of course UDL principles, those are Universal Design for Learning, and that does include equity in its heart. So that would definitely help to keep things equitable in the classroom. If you’re following UDL.

Rebecca: The multiple forms you were just talking about is a great example. [LAUGHTER]

Judie: Last semester, I had a student who, they’re supposed to do a blog post, and the student instead of writing a blog post, he made a video and he did it three different times. So one is on World War One, one’s on World War Two, and the third one was on revolutions, and so, this student stood in front of a whiteboard, and he had his camera set up so he could film himself. And he had his iPad in his hand. So he talked about a battle, say, for example, and he would draw it out on the board. And then he would show his citation on his iPad. And then he had other citations typed up and taped to the whiteboard. And he went on for 15 minutes, and just was making sure he explained things again, and drew little examples. And he was so animated, and so excited about his topic. And you’re not going to capture that on a written exam, or even in somebody’s written paper. It was just tremendous the way he was able to show all that he had learned and all that he was interested in, and the extra research that he had done, because he felt the freedom to pursue this topic, because he knew he was able to express it the way that suited him the past. And it was just amazing. So I think anytime we can incorporate these things, and I understand that there are times when, according to your creditor, or people have to sometimes sit for a specific certification, it doesn’t always fit, but I think if you can fit this type of assessment in, it is definitely worth it. Because just to see the joy in students when they can explore and expand their knowledge, and then feel confident in demonstrating that to you, it’s just tremendous.

Rebecca: I love the flexibility in demonstrating knowledge and understanding and skill sets because in some of our traditional methods, we are arbitrarily assessing something else. So we may be arbitrarily testing how well you can take a multiple choice test or how well you can take a test within a certain timeframe, or how well you can write, whether or not that’s actually the topic. So if I’m learning about history, there’s some learning objectives I’m trying to meet related to history that may or may not include writing. And if writing is not one of those outcomes that we’re hoping for, then we don’t need to be assessing it.

Judie: Exactly. He did this thing on medical advancements in World War One, it was just tremendous and he was so charming, because he just was so wrapped up in it that you just had to root for the guy. It was good.

Rebecca: I love that. So for those of us who may want to move towards equitable grading systems, what are some initial steps we might take? Because it could feel really daunting if you haven’t ventured down this path before.

Meghanne: Yeah, if you are not interested in overhauling your entire grading system, just to try this out, a nd to make your assessments more socially just, there are some adjustments that can be made to existing assignments. And really, the important thing is to consider the learning objectives and really think about what needs to be graded. So one of the things that we’ve talked about a lot in all of our different presentations that we’ve done is whether or not to grade for things like grammar and spelling, and mechanics, and English language proficiency. So in an example, like a discussion board, when you’re really interested in what the students have to say, and their interaction with each other, and the questions that they asked, does it really matter if their grammar and spelling is perfect in that instance, if they’re having a great conversation on a topic, and they’re learning from each other. So that’s one thing that we could suggest. Another is thinking about just the fact that sometimes students have challenges in their lives. They’re human beings, they have families, they have jobs, many of our students are athletes, and they have to travel and they have games and something like flexible due dates is very, very helpful for students because then they’re able to complete their work, certainly within a reasonable timeframe. But if those dates are a little bit more flexible, and they have access to those assignments in the learning management system beyond the actual due date, for instance, then that gives them the ability to complete that work without being penalized. So another mechanism would be in the learning management system, when students are taking quizzes, would be allowing backtracking, allowing students to go back and check their answers, that sometimes is a setting that a lot of professors really rely on, to try to avoid cheating. And as an LMS administrator, that is something that I see a lot. And I think that that can really be harmful to students, because many of our students are told to always go back and check your work. And if they’re not allowed to go back and check their work, that can be very frustrating. And also forcing completion is something that I would recommend turning off because again, that can create test anxiety. And often I think when completion is forced, there’s also a timer. So I think if any timers can be removed as well, then that does a couple of things. It can help remove testing anxiety. But then also, if there are students who require extra time due to a disability accommodation, then the professor at that point doesn’t have to go in and adjust all of the LMS settings for those students, because it’s already open ended and everyone can have as much time as they need to complete that assessment. So it really is just important to look at what the learning objectives are and what actually needs to be assessed. And the goal is always to remove barriers. So another thing that can be done is to just ask students, have a conversation about it, and find out what barriers they’ve experienced.

John: At the start of this. You mentioned the website that you were creating, could you talk a little bit more about what resources are there and how that might evolve over time?

Katelyn: Yeah, so the website, we have been slowly adding resources to over the past two years. And at this point, it’s becoming a pretty robust little outlet for people interested in social justice assessment. So, you go to the website, you can find an overview of the big picture theory of social justice assessment, as well as the various approaches that we’ve discussed today. We also have a really pretty large bibliography of resources for further reading for people who want to learn more about any one of these topics. And we’ve been working to develop a collection of sample assignments from faculty across SUNY. So we’re still working to collect additional sample assignments from faculty who might already be implementing some of these strategies within their classrooms. I think the more we can share those assignments with one another, the better off we’ll all be. I think a lot of us are doing social justice assessment in small ways in our classroom without realizing it. So the more we can share those resources and that knowledge, the more hopefully we can get people on board. So, hopefully, we’ll be able to share that link in the show notes. And people will be able to check that out.

Rebecca: So we always wrap up by asking: “What’s next?”

Judie: So for our little group, one thing that I think might be next for us is SUNY is updating the SUNY general education requirements that are mandated with the completion of any SUNY degree. And they’ve added a requirement for equity, inclusion, and diversity. So I’m hoping that our group can help contribute resources to that effort, and our website could be one more place where people go to for information on social justice assessment so that they can incorporate those into their courses that are designed to meet the DEI requirement.

Katelyn: Well, I’m gonna go take my one-year old to the pool. [LAUGHTER]

Judie: Nice.

Katelyn: I think, big picture, though, the “what’s next” I want to just give is, I hope that we’ll start to see more institutional support for some of these approaches. I think that there are still a lot of barriers, particularly for contingent faculty who want to embrace some of these practices. So I hope what’s next will be more departmental institutional support for this: more time, more resources, etc. But yeah, my personal what’s next is I’m gonna go enjoy this beautiful day.

Rebecca: Meghanne, do you want to add anything?

Meghanne: Sure yeah, at my institution, I am sharing this information, pretty much any chance I get, I’m meeting with our new incoming faculty in a couple of weeks. And this will be one of the topics that we discuss. And I’m also co-chair of our universal design for learning task force. And we have a few events and projects that we’re working on to spread the word on UDL, and also innovative assessments and social justice assessments as well.

Rebecca: Lots of great things coming and some really wonderful resources that you’ve shared today. Thank you so much for joining us.

Katelyn: Thank you.

Judie: Thank you for having us.

Meghanne: Yeah, thank you.

John: And thank you for all the great work you’ve done on this over the last couple of years and the resources you’re sharing.

Judie: I would just like to say that Shena Salvato is also in our group. She’s at Cortland, I believe. And Chris Price from SUNY is in our group, and they are missed today. They’ve been with us for all our other presentations. I know that Shana in particular wants to get the band back together and have some more meetings going forward so we can keep working together. And it was really good to see you guys again.

Katelyn: Likewise.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

250. Hacking Assessment

Traditional grading systems often encourage students to focus on achieving higher grades rather than on their learning. In this episode, Starr Sackstein joins us to discuss how classes can be redesigned to improve student engagement and learning. Starr has been an educator for 20 years and is currently the COO of Mastery Portfolio, an educational consultant, and instructional coach and speaker. She is the author of more than 10 books on education, including the best-selling Hacking Assessment: 10 ways to go gradeless in a traditional grades school, which has just been released in a new edition.

Show Notes

Transcript

John: Traditional grading systems often encourage students to focus on achieving higher grades rather than on their learning. In this episode, we discuss how classes can be redesigned to improve student engagement and learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Starr Sackstein. Starr has been an educator for 20 years and is currently the COO of Mastery Portfolio, an educational consultant, and instructional coach and speaker. She is the author of more than 10 books on education, including the best-selling Hacking Assessment: 10 ways to go gradeless in a traditional grades school, which has been released in a new edition. Welcome, Starr.

Starr: Thanks so much. I’m excited to be here.

John: Today’s teas are… Starr, are you drinking tea?

Starr: I am drinking water. No tea unfortunately, not yet.

Rebecca: Not yet. Okay. See, there we go, there’s promise there. I have Scottish Breakfast tea today.

John: And I have spring cherry green tea.

Rebecca: Well, that’s good.

Starr: Those both sound delicious, really.

Rebecca: So, you haven’t had that one in a while, John.

John: I haven’t had any in a while…

Rebecca: true that…

John: …we took a pause in recording for about a month. [LAUGHTER]

Rebecca: Fair. But even prior to that it had been a while I think.

John: I think so too.

Rebecca: We invited you here today to discuss Hacking Assessment. The first edition of your book seven years ago helped to launch the ungrading movement. Could you give us some background on what prompted you to move away from traditional grading systems?

Starr: Absolutely. In years one to five when I was in the classroom, I would say that I pretty much did grading and assessment the way it was done to me. And the one major significant thing that changed during that time was I had a child. And in his elementary school, they actually use standards-based grading. And when I got his first report card and saw just how much information I got from his teachers, and how the behaviors were separate from the actual learning and the narratives were really aligned with where he needed support and what was going on. I was like, Mmmh…for someone teaching AP English, only having the opportunity to give one grade, with pre slugged sort of comments that I was allowed to bubble into my… back then we were still using Scantrons for entering grades. I’m definitely dating myself by saying that, but it’s the truth. And I started getting really frustrated with that. And from there, I started doing a lot of reading. Alfie Kohn has really played with a lot of these ideas for a long time now. And then folks like Ken O’Connor, who had the book 15 Fixes for Broken Grades, his first edition, I think it’s been republished twice already, in the time since I’ve read from there. I read his book, and I was like, “Oh, my God, I am doing all of this wrong.” There are so many things on this list that I do, and I never thought about it that way, and it’s just not how I want to keep doing things. And I think there’s a synergy with when you decide to read a book, whether or not it resonates with you and whether or not you’re ready to start implementing the things that you learn. And I think I was very ready to first acknowledge that the practice I was doing wasn’t serving my students as well as I could. And I was looking for alternatives. So having those jumping off points, having read a bunch of different things, and then meeting Mark Barnes along the way as well, and experimenting with alright, well, these are suggestions for this kind of space. What does this look like in New York City public schools, as an 11th and 12th grade English teacher and also as a journalism teacher? How do I start making this work? And that’s sort of how it all happened and then it took years to figure out how do I make this work well, because I did it for a while before it worked well. [LAUGHTER] There were a lot of mistakes, unfortunately.

John: We’ve been dealing with a number of people starting to experiment with ungrading in college, but it’s a little bit easier in a college environment, I think, to make these changes, because there’s a little bit less structure imposed on teachers. How were you able to implement this in a K through 12 system?

Starr: So I think I was very fortunate to be in a very small community when I started doing this. We were six to 12. I was already a very established teacher in that community. I had a track record of getting students prepared for college. And most of the families when I made choices, always kind of knew that they were intentional, and there were reasons. And in my AP classes, that was probably the most struggle, because parents get nervous when they have 12th graders, what is this gonna look like on the transcript? How is this going to impact my students moving forward from school? And I just really tried to set up systems and to be super transparent about everything that we were doing so that first of all, I live streamed my class a lot, for better or for worse. And I say that because not every class was a winner. So if you were watching when it wasn’t a winner, like, well, this is reality, it wasn’t a good day. But I think they were able to see the rigor of what was going on in the space and despite the fact that it didn’t look like what normal AP classes looked like, they could appreciate my wanting to be flexible to the individual learners in my classroom… that even the creative projects I was asking them to do was often a lot more intensive than just doing a test or just writing a paper and gave that level of inquiry into that process as well so that students could be really excited about the learning they were doing. And the more comfortable I got with different technologies… I experimented with blogging to increase reading. That’s one of the biggest problems in English classes. I think most kids don’t read the books for a lot of different reasons. So how do you get them to read when you’re teaching a literature class, beyond just the five or seven or 10 books you’re reading as a whole class. So they started blogging, and we started using the blogging communities for recommendations on different books they were enjoying on their own and why they enjoyed it. And I really encouraged them to use that space too as a way to develop their writing voice. So it wasn’t like analytical writing all the time, it was more conversational… reaction sort of stuff to what they were reading and focused instead of like overviews of everything that they read… an analysis paper, which isn’t always fun for every single kid. I started tweaking that and I think parents appreciated my transparency. I did screencasts of our dashboard, because I had changed the way I was using the tool that my whole school was using. So like, if you have any questions, this is what it looks like, this is what you’re seeing. And if they emailed me, I just really tried to get back to them immediately, so that I could really put their concerns to rest before they started doing the thing that parents do, where they start making it a lot worse than it actually is. So I tried to catch that right away. To be honest, though, my colleagues were the ones with the greater pushback than parents and students… a couple of students, but just shifting the conversation away from grades, instead of what did I get? What did you learn? How can we track that progress over time? How do you know you learned it? Where do you see that evidence in your own learning. And I think very soon after getting in the routine of this is how we do things now they got it and saw that the level of metacognition as well as the rigor in the actual tasks were much greater than what they would have been seeing in a regular class anyway. So sometimes I got the: “this writing reflection is like a whole other paper that you’re asking us to do.” And I’m like, “Yeah, it is. But it also helps me give you better feedback, and it also helps me know where I need to adjust my instruction. So there’s a reason and it’s worthwhile, and it’s gonna help you, when you’re not just in school. This is a practice that you’ll probably carry with you.”

Rebecca: One of the things that you just brought up, Starr, is something that I definitely want to follow up on, which is getting our colleagues to also buy into this, and administrators. We exist within systems that require grade inputs. Grades are transcripted. So how do we get the people around us who support us professionally, to get on board? And what does that actually look like, functionally, when we’re generating grades when we’re saying we’re kind of ungrading all semester.

Starr: So those are really good questions. And in the second edition, I actually have built in leadership tips to support leaders who are unfamiliar with this kind of assessment practice and how they can support teachers who want to do it, if they’re not doing it wholesale as a school. I advocate for systemic use of this practice, because if we catch kids much younger, by the time they get to high school, their language and fluency in discussing their own learning is a lot greater. I was a 12th grade teacher, my kids had come through an entire system where this was not how it was done. So it was like, literally at the last minute, I’m like, “Yeah, I know, that’s the way you’ve been doing it this whole time. But we’re gonna do it a little differently and I promise you’ll still get into college.” It’s a different vibe. And my colleagues, I think, knew my students appreciated it, because they would start hearing from my students: “How come you don’t do this?” …which is also like a little bit of a target was put on my back, because if a school or a district is going to make the shift, it requires a lot of professional learning. And if you aren’t the kind of teacher who makes the time to do learning on your own, then there really does need to be supports put in place prior to it’s happening. And I’m a super reflective teacher, I did National Board Certification, I will go out of my way to get myself to a conference even if my school wasn’t paying for it. Because as an educator, I felt it was an essential part of my job to continually grow and model that for my students. But not every teacher is like that. And I’m not suggesting that everyone has to be or whatever their process is, but I do think it’s important to invite colleagues into your space, give them that “what you could do tomorrow kind of tips” like what are the first few steps you could take to try this out before you commit to it wholesale. And in terms of the grading aspect, the way that I got around the traditional grading was assessment conferences with my students. So really building in a vibrant and robust portfolio system where students were collecting their learning over a larger period of time, giving them the vocabulary to talk about their growth as they looked at those things, and then a conversation just like this. So based on the standards we worked on this marking period, where do you find yourself in terms of mastery? And what does that translate to for a report card grade, because I had to put a grade on the report card as well. So it was really just making them acutely aware of what exemplary work looks like, how they were meeting benchmarks to get there over time, and then also switch that transactional sort of relationship around getting grades to a more progress minded model, where they understand learning doesn’t happen in one sitting. And even though you may have successfully completed one assignment, that doesn’t mean you’ve mastered a particular skill, it’s just your first go at it. In order to get to that mastery level, you have to do it over time with less and less support, and kind of do it on your own.

John: What sort of buy in did you get from other teachers that you were working with?

Starr: It was secret at first. There were like people just dropping by out of curiosity to see what was going on in my classroom. Then a couple of other people just asking, “what would this look like in my gradebook?” I was very lucky in the one sense that our whole school was a portfolio school. So that part of it was already there. And then I also did some PD with my colleagues around reflection practices. We tried to really create something that was consistent, and also the same. So like I had created a process for doing reflection, which is that five steps sort of: first, you have to reexamine what was it I was asked to do? What were my steps for completing the assignment? Where do I think I’m meeting the goals that I set for myself? How am I doing that? What level am I doing that at? And what would I do differently in the future? And then we kind of scaffold that down to sixth grade up to 12th grade. So what is that kind of reflection look like in a sixth grade classroom, a seventh grade classroom, all the way up to 12. So that there are realistic expectations in that space around those things. And my classroom was always open. And I resented the fact that when my principal decided that she wanted us to go to a standards-based model, I implored her to not do it the way she did. I think we should have a pilot team, we should have a committee that does this, we should test it out first, try to get either a grade level team or a content area vertically to commit to doing this and then have input from more people. And then we need to train folks in the areas they aren’t already familiar with, starting with unpacking standards and getting them comfortable with that kind of language and what our expectations are. But that’s not what happened. It was like an email that went out. We’re going to do this this year. And it was a disaster. And I got attached to the disaster as a direct correlation to how all that happened. And unfortunately, you get one good shot to make a significant assessment or grading shift in a decade, because unless your folks are leaving quickly, no one forgets. So really setting up systems in the future, if folks who are listening want to do this on a bigger scale, set yourself up for a three- to five-year implementation plan, start small and grow it organically and provide tons of support along the way so everybody feels confident and not just your teachers, your community also. What does this look like for your parents? What are they going to be receiving that’s different? And just make sure that you have answers to commonly asked questions on the front end, so that when new stuff starts coming in, you’re ready to triage that, you’re not just answering the standard questions over and over and over again.

John: You mentioned in your first edition of the book that one of the motivations for this was to get students to focus on their learning rather than on grades. How successful was this? Did this work for most students?

Starr: For most, yes. And believe it or not, the ones that don’t traditionally do school well, who don’t play the game, it worked best for them. And as three educators sitting on this podcast right now, I think we can all agree that sometimes our brightest students are not the ones who do the best. The ones who do the best are the ones who are most committed to getting high grades and kind of checking the boxes and doing everything that they have to be compliant for in order to get that score. So when we shifted the focus away from that and started looking at skill acquisition and content deepening, and really getting them to be able to advocate for their own needs in that specific area, I think that it wasn’t just about them completing the tasks I asked them to do, but it required them to engage with me in a dialogue in the kinds of tasks they wanted to be doing, the way they wanted to be doing it. And it required my flexibility with taking that input and actually putting it into action. So I think that once they saw that I was listening to their feedback actively and using it right away to shift the way class looked, they understood that I wasn’t just saying, “I’m asking you to do this,” it was a real partnership, where if this is going to be successful, and you want your voice to be heard, you need to contribute or else you can’t complain when you don’t like what ended up happening, because I really did try to say “yes,” just about to everything, if they could articulate how their decisions and their choices aligned with what the objectives were, then I was totally hands off in their process to sort of help them be successful in the big picture. And it also really decreased the amount of folks who didn’t participate in the group work or didn’t participate in the learning. So when people say my students don’t finish work, or they don’t submit things, to me, that’s a red flag that either something else is going on that you need to get to the bottom of, or the kind of learning you’re asking them to do isn’t resonating. And rather than just pulling out the binder from what you’ve done for the last 20 years, you really do have to make a concerted effort to make changes so that it meets the needs of the kiddos that are sitting in front of you right now.

Rebecca: So you’ve talked a lot about reflection, and the role reflection is playing. Can you talk a bit about how you were able to get students up to the level of reflection that is really meaningful and gets to this metacognitive skill, building

Starr: Feedback, feedback, feedback. We give a lot of feedback to everything that kids do in the classroom. But the first few times we ask them to reflect, it’s so important that we’re also giving them feedback on their reflections, providing exemplars for them, really creating success criteria too, like that co-construction, like if I’m telling you, these are three examples that are wildly different, but all successful, what do you notice about all three of them? What are the things that need to be a part of every single reflection that we do. And then as they do them, rather than have them revise every single one that they do, since they’re doing them with every major assignment, it’s like, “alright, well, now take the feedback you got from the last one, apply it to this one and let’s see if we can’t grow you.” And usually by, I would say November, they’re already writing fairly good reflections and their ability to have conversation about their level of learning already starts to increase, because by November, you’ve already had a progress report conversation, you’ve already had a quarter one report card conversation. And I was doing a lot of modeling myself, like I would reflect openly on how successful projects went, in my estimation, and be really, really tied to the outcomes. And not just what I think or what I feel, but what I noticed, and how I would do it differently if we had the opportunity to do something similar again. And I think, again, that level of transparency and my comfort with saying to them, I don’t know how to make this better. What do you think? What made this experience challenging? Were my directions not as clear as they could have been? What do I need to learn from this experience? So it was very much a two-way street, which took time. And I do want to say that too. Like, I think I was seven or eight years into the classroom before I was comfortable enough to say “I didn’t know something.” That takes confidence in a way that you don’t really think. In the beginning of my career, I felt like I needed to be the expert over all of the students in my room, and I had to have an answer for everything. And I said a lot of wrong things because I was trying so hard to look like an authority. And I think the older I get, the more I work with educators, the more I realize that I’m a learner, I don’t know everything, even the stuff I’ve spent a lot of time teaching I don’t know everything about and new perspectives are incredibly useful in how I approach something because it’s the first time this group of kids is seeing something I might have tried before. Their input is extraordinarily useful for me to make changes moving forward.

John: It’s also a great way of nurturing a growth mindset in students by reminding them that we’re all part of this learning experience together. And that no matter how much experience you have, there’s always more you can learn. And so I think that’s a really great process. And it’s something that I think it generally takes a while for most people to get to.

Starr: Yeah.

John: So you mentioned having conferences with students, how often do you conference with students?

Starr: So, there’s lots of different levels of conferencing. So you have your in-class formative conversation where they’re asking questions and you’re taking the pulse of whether or not you’re going too fast or if you need to stop the class and do a mini lesson on something you notice everyone’s struggling with. Or if you pull a small group because only a small group of kids are really having an issue. So there’s that kind of on-the-fly conferencing where you’re walking around with a clipboard or an iPad and you’re taking notes on what you see. And then listening to the questions kids are asking and making a determination as to whether or not this is a small or bigger issue that needs to be addressed. And then there are formal conferences where kids are coming prepared to have that conversation where you’re giving them time in class. So part of my structuring… because remember, I said it took me a long time to find a system that worked that ended up in Hacking Assessment… so I started creating Google Forms, where there were very targeted questions that also aligned with the assessments that we did, and the different pieces of learning and the standards that we were addressing at that time. And before they could set up a conference, they needed to fill in that whole Google form, then I had all that informatio, so I could really target clarifying questions or gaps that we could spend our five minutes talking about. If they had done all the work to do certain things, they don’t have to rehash what I could read. And if I had 34 students in most of my classes, so there’s a lot of kids, there’s a little time, you really have to make that three to five minutes count, and give every student the opportunity to give you the most information that you could have to be able to determine what was going to go on the report card. So those conversations certainly got a lot better over time as well. The first one, there was a lot of prompting from me, a lot of questions to get them ready by conference number 2, 3, 4, and certainly by the end of the year, if you watch on my YouTube channel, I have examples of what those look like. By the end of the year, the student is doing 98% of the talking. And I’m just redirecting if they kind of get off a little bit, or if they miss a spot versus at the beginning, it’s more of like a 40-60 where I am interjecting and kind of bolstering confidence, helping them set goals and stuff. So there’s more of a give and take at the beginning of the year.

John: You mentioned giving students some choice in terms of the assignments and so forth. What are some of the more interesting assignments or learning activities that your students have come up with?

Starr: The one that always comes to mind was, towards the end of my time in the classroom, before I became an instructional coach, I literally gave my students my entire unit plan for Hamlet. And I said, “Alright, this is the way I always teach it. But I want to do it differently this year. So I want you to look at the overall objectives. And as a group, I want you to come up with something different, then we’re going to vote as a class, which group suggestion we want to go with, and whichever group is chosen, you’ll come meet with me at lunch, we’ll design an assignment together and work through the success criteria and benchmarks for doing it successfully.” And if I tell you some of the things these kids came up with, I would have never come up with in a million years. And what we landed on was these psychological profiles of the characters of Hamlet, where they had to first use the text, to use Shakespeare’s language, to diagnose them with some kind of psychological issue. For example, Gertrude would be a narcissist. And then they do research on the actual issue, so there’s a research component as well. And then they had to come up with a treatment plan for the character and create a movie that demonstrated the growth from whatever the treatment plan was. And what it really did was have this really in-depth character analysis of each character from Hamlet, regardless of which character you did, you were set on a course. And then we also created this Google form, so that when we had screenings of the movies at the end, students were actively taking notes about what they learned about the characters and giving feedback at the same time to the creators of those movies about what they learned and what they were still curious about. And it was really phenomenal, honestly. I think that I wish I would have started doing stuff like that sooner. Other examples would have been students creating movies in Minecraft, like for our satire movies, that’s usually so like, just technology, but I was very uncomfortable with, that they were able to use that. I was like, yeah, “If you could do it without my support, I could help you with content, but you’re on your own for the technology.”

Rebecca: So you’ve hinted at some of the changes in your second edition. Can you highlight some additional changes between the first and second edition?

Starr: Okay, so yes, there are a lot more resources. So over the last seven years, part of the reason I hadn’t made a second edition up till this point, was because I really wanted there to be a value added. I wanted there to be new voices I can highlight. I was really also looking for systems that started doing this work because I wanted there to be more case study material that kind of went in that it wasn’t just single teachers kind of playing with it, but actually systematizing it in ways that work for them. So there are brand new hacks and actions for every single chapter, all of them have read the first edition and implemented it in their own way. So what you’re getting is people’s take on how what they learned looks like. I really tried to implement K to higher ed. So Susan Blum did write a section as well on what it looks like in college for all of my reticent K-12 folks who were like, “This isn’t going to be viable in the future.” I had central office people write about stakeholder buy in and how they brought this into their space from a leader perspective, instead of just a classroom perspective. A lot of new tools that have been developed in the last seven years, lots of stuff about that, rubrics, progressions, not just in English, which was my background, obviously, really trying to span math, science, social studies, related arts. So there’s one with a music teacher writing about how they’ve done that in that area… elementary teachers. So there really are tons of resources with a lot of different fresh voices who are using this now, as well as a very intentional talk about equitable practices. I think a lot of this stuff is equitable, but I never thought of it in that lens until COVID. And then once COVID happened, really trying to talk about how these things address some of those gaps that need to be addressed, but weren’t explicitly tied to them in the past. So that’s really where the bulk of things have shifted. And then there’s an incredible appendix with lots and lots of examples of everything.

John: And your first edition was wonderful. It provides a lot of good resources. And in each section, it talks about how to deal with pushback, which is one of the things anyone introducing something new has to deal with. So I’m assuming that continues into the second edition.

Starr: Yep, sure does.

John: So your first edition was very successful, and has received a lot of traction at all levels of education, and helped spur the ungrading movement at the college level that we’ve been talking about a lot in the last couple of years with our guests, and with many of our colleagues. For those people who have read the first edition, what would be the benefits to them of picking up the second edition, and who should they share that with at their institutions?

Starr: So I’m really hopeful that this time, it’s not individual teachers picking the book up on their own, although I certainly advocate for that. I want to see teams use this as a PLN opportunity and explore the text in a way that makes sense to them. It is not narrative, necessarily. So each chapter is its own sort of entity. And so I would encourage folks to choose the chapter that they’re most ready for at this moment and pick it apart in a way that’s going to make most sense for their practice.

John: We always end with the question: “What’s next?”

Starr: Oh, I’m so glad you asked. So what’s next for me right now, we are doing a free book study with the new book when it launches August 2, and it’ll be on Amazon. And then also, once this one launches, and things are moving, I’m under contract with ASCD for my next book, which is specifically about portfolios and student-led conferences. So that is still something that’s a little thinner in Hacking Assessment, because I think that that really requires a little bit more depth than I could give it in that book in one chapter. So I am currently working on that and really trying to gather with some of the districts that I’m working with to build really great systems for building portfolios. What does that look like? And how do you parlay that piece into these student-led conferences so that you can have a robust system in your space?

John: That sounds like a great supplement. Well, thank you. It’s great talking to you. We’ve heard mention of your book from many of our past guests, and I’m glad I was finally able to get to read it. And I’m looking forward to the second edition, which should be arriving soon.

Starr: Awesome. Thank you so much.

Rebecca: Yeah, thank you so much. This is such great information and we’re looking forward to all your new work as well.

Starr: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

244. Unlearning

To deepen our understanding or improve our skills, it is often necessary to question our preconceptions and unlearn some of our past practices and assumptions. In this episode, Lindsay Masland joins us to discuss her unlearning journey. Lindsay is an associate professor in the Department of Psychology and the Associate Director of Faculty Professional Development in the Center for Academic Excellence at Appalachian State University.

Show Notes

  • Blum, S. D. (Ed.). (2020). Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press.
  • Jesse Stommel’s website
  • Stommel, J. (2018). How to Ungrade. Blog post, Jesse Stommel. March 11.
  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.
  • Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087.
  • Hattie, J., & Clarke, S. (2018). Visible learning: feedback. Routledge.
  • Pittman, C., & Tobin, T. J. (2022). “Academe Has a Lot to Learn About How Inclusive Teaching Affects Instructors.The Chronicle of Higher Education. February 7.
  • Chavella Pittman and Tom Tobin (2022). Include Instructors in Inclusive Teaching. Tea for Teaching podcast. Episode 231. March 16.
  • Prentis Hemphill

Transcript

John: To deepen our understanding or improve our skills, it is often necessary to question our preconceptions and unlearn some of our past practices and assumptions. In this episode, we explore one faculty developer’s unlearning journey.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Lindsay Masland. Lindsay is an associate professor in the Department of Psychology and the Associate Director of Faculty Professional Development in the Center for Academic Excellence at Appalachian State University. Welcome, Lindsay.

Lindsay: Hi, thanks for having me.

John: Are teas today are… are you drinking tea, Lindsay?

Lindsay: I am not because I’m one of those people that can’t have a lot of caffeine in the afternoon hours and this is afternoon hours for me, so I am drinking store brand seltzer. So, very fancy.

Rebecca: It sounds very fancy to me. [LAUGHTER] It sounds perfect. I’m celebrating the fact that it feels like it’s a summer day here, which is magical. And so I made iced tea fresh.

John: And what type of iced tea is this?

Rebecca: This is English Breakfast iced tea.

John: Very good.

Rebecca: Decaf.

John: Lady Grey, by the way, makes a superb iced tea.

Rebecca: It does, you’re right.

John: I had that for the first time at the English P=avilion in Epcot when we’re at one of the OLC conferences, and I had to ask them what the tea was because it tasted superb. I had never had it as an iced tea before.

Lindsay: Sounds like I need to branch out because I am in the south, you know, and we do like our iced tea. And I mostly have iced tea that I brew the tea myself, but it’s always English breakfast tea. I hadn’t thought to branch out.

Rebecca: Yeah.

John: Since this was an English pavilion, it was probably acceptable to try that.

Rebecca: You didn’t say what kind of tea you were drinking, John.

John: I have a black raspberry green tea today from the Republic of Tea.

Rebecca: Oh, that sounds nice.

John: It is good. I haven’t had it for a while.

Rebecca: I don’t usually think of raspberry and green tea together. I always associate that with black tea. That’s all… That’s the whole thought. [LAUGHTER] There’s nothing more there. And welcome to our land, Lindsay. [LAUGHTER] It’s the end of the semester. So we invited you here today, Lindsay, to talk about unearning. Can you talk to us a little bit about what unlearning is?

Lindsay: Well, I guess I should first say this is not my term. There are probably other people who could more likely say that this is their term. But I just know that when I think about my own experiences as a faculty member over the last 11 years (that’s how long I’ve been in academia), and then a faculty or educational developer for the six or so, that the main thing that’s happened for me is recognizing how much that I used to believe was true, that simply wasn’t serving me anymore. It wasn’t serving my students, for sure, but it also wasn’t serving me just as a person. And so to me, that’s what unlearning is, is when you have those aha moments, the achievement of threshold concept moments, if we want to connect to some of that language from faculty development, where you realize this is not something that I want to continue to believe or live out.

John: And I think this also applies to our students’ experiences and to our role in teaching students… that they come to us with a lot of preconceptions, as we come to teaching with a lot of preconceptions, and some of those don’t hold up very well. So I think it’s a great topic to be discussing. What are some things that you have unlearned, since you’ve been in this role?

Rebecca: Or are in the process of unlearning? [LAUGHTER]

John: Yes.

Lindsay: Yeah, I think that’s a really important caveat is that I feel like we never arrive, I think as teachers as humans. And so we’re always in the process of doing something I think a lot of the times, we’re thinking about being in the process of learning. But simultaneously, I think we should be in the process of unlearning the things that don’t serve. So I think one major thing that I was kind of socialized into is… my background is Psychology, my PhD is in Educational Psychology with a concentration in Quantitative Statistics, and I only bring that up because I think it’s really important for understanding, I guess, the baggage [LAUGHTER] that I brought in some ways to the teaching role, because psychology, especially when I was getting my PhD and the time before that really overprivileged a quantitative “objective” view of the world. And so that is what I was socialized into. We didn’t learn any qualitative analysis, for example. And so that’s kind of what I was carrying with me. And I also think… I don’t want to blame it on psychology, I see why psychology is that way… because psychology at one point kind of split off from philosophy hundreds of years ago. And one way they were able to distinguish themselves from philosophers was to say, “Well, we have science and we collect objective observations about things that feel really non-objective, because psychologists study feelings and behaviors and ideas and thoughts… things it seems like you shouldn’t be able to quantify those.” But like, our whole shtick is that we can. And so I bring that with me, I think, into the teaching, into educational development. And I don’t think that was very helpful, [LAUGHTER] necessarily, because what I think ends up happening is you start, at least I know I brought kind of a deterministic way of thinking, like, “Okay, we just got to figure out what are the evidence-based teaching strategies, and I’m going to learn those, and then once I become a faculty developer, I’m going to teach those. And then if we all just do that, everything will be great.” Because that’s a very, like, if then we measure this, we do this, we get this clear result. That’s the whole thing with psychology is trying to predict behavior. And so if you bring that into the classroom, it’s like, I’m going to design in a way that’s going to predict everybody’s behavior. So I think that was something I needed to unlearn. When I realized, you can’t predict behavior, [LAUGHTER] that teaching choices are not deterministic, they are contextual, and that you really need to bring some chaos theory [LAUGHTER] into your understanding, honestly, of teaching. So I think it was when I started to read some books seriously about chaos theory, and also about different types of statistics that were intentionally modeling either context or randomness. And I was like, wait a minute, we’re taking a math equation, and we have like a thing in the math equation that is measuring “randomness.” I was like, what’s that? But it made me realize, like, wait a minute, okay, if the physicists and the mathematicians are doing that, we need to get on board.

John: This really resonates with me, because my background is that I’m an econometrician. And I got interested in this by doing some research on what techniques seem to work in my classes and in other classes. And those error terms, though, I’ve always taken pretty seriously. But in recent years, I’ve become much more interested in behavioral economics, which introduces all the ways in which we don’t behave in ways that are entirely consistent with the economic models that we normally teach in our classes. Actually, I’ve been bringing in more psychology into economics, which is probably even more deterministic than psychology ever was.

Lindsay: Yeah, that’s a good point. But it’s so funny, because in so many of these models, we’re always trying to reduce the error term, or control for the error term, like make it irrelevant by our methodology, we’re going to control for the error. And I think once you start teaching, you’re like the “error,” first of all, that’s like a really problematic label [LAUGHTER] for your students, but the “error,” it’s like, that’s where it’s at, like, we need to actually be designing for that, because there is variation in ourselves and in our students. That’s, I think what successful teaching is. But, it’s important, because I’m seeing this discussion right now a lot on social media is people are expressing their, what I would call righteous frustration about certain teaching strategies, assessment strategies, belongingness strategies, any of that… certain strategies being kind of like heralded as the ideal, and then other people responding like, “Well, I can’t do that. That’s not feasible for me. So does that mean that I’m not going to ever be a good teacher?” Because that’s not effective. And to me, I’m like, “Okay, this is just all playing out now in our discussions, is people are starting to embrace the idea of context or interactions.” I mean, I think about it in terms of statistics. And in statistics, we have things called main effects where there’s one variable, and it seems to affect everybody equivalently. But most of the time, there’s also an interaction where different variables are interacting. And anytime there’s an interaction, you pay attention to that thing, not the other thing. And so I’m kind of excited that people are getting angry about “Well, I can’t ungrade…” …for example, or something like that… things that are a lot of people are talking about, I’m like, “Well, good, let’s have a discussion about that.”

Rebecca: As an artist, I really appreciate you coming to my site.

Lindsay: Actually, this connects well to unlearning. My day job, I guess we can call it, is an academic. But I have a second kind of life after my day job, which involves being on the stage, I have a lot of theater and dance activities that take up my time. So I actually have this whole artistic side of myself. But one of the things that I was kind of socialized into was keeping those separate. Academia is for serious people and art is not serious. And I want to really make it clear that I don’t believe that, but like, that’s what I was socialized into. And then it was a big unlearning, honestly. I stepped away from theater and dance for like 10 years, when I was finishing my PhD, and up until getting tenure. And in retrospect, I think I was doing that to be more serious. I was always saying to students, I get tenure, I’m going to do a musical again. And I did it. And I’m even getting emotional thinking about it. I was like, how did I live without this part of myself for 10 years, and that was really damaging. And I was like, I shouldn’t have had to do that. And I don’t want anybody to have to do that anymore. So, I think it’s really important because that was damaging to kind of live without that side of myself and now they’re together and I feel like a whole person again, because I’m bringing the subjective and the objective in together.

Rebecca: I think it’s important to remember that we’re all human. And that often gets lost when we’re thinking about teaching or we’re thinking about scholarship or we’re thinking about a trajectory. As a faculty member, the humaness and the things that are outside of the Academy are often lost or not attended to, at least during that tenure process.

John: And I had a very similar experience in terms of playing music up until the time I was in grad school. My band got together and went on tour, and I stayed in grad school. It was probably 15 years or so before I started playing again, and it’s gone back and forth a few times. But it is much more interesting when you can be that full person. And going back to the analogy with empirical studies, most of the variation in most studies of teaching is in that random component. And those random components are the people, the instructors and the students in that relationship. And it’s important not to forget that. So this is a really good point, it’s really easy to forget in our day-to-day work. So it’s good that you’re focusing in this direction, I think we all probably should focus more on being that whole person, especially now.

Rebecca: And we’ve all had those experiences of that randomness, because you might have two classes that feel entirely different, but it might be the same subject, the same syllabus, the same teacher, but the students in the room are different, the time of day might be different. So therefore, the context is now different.

John: What are some of the other things you have unlearned?

Lindsay: So another thing that I guess kind of follows on from that unlearning the obsession with objectivity is also disentangling or coming to understand what is my proper role in the classroom. And so starting in academia, I think a lot of people have this kind of experience, I guess I was 29, maybe, or just turning 30, when I was in my first tenure-track position, which is the position I’m still in, I’ve been at the same institution the whole time. And so I’m a female, I’m still in the same decade as some of the students that I’m going to be teaching. And I also am somebody that has a young face. I’m not tall, kind of like pint-sized, in some ways. [LAUGHTER] And so I have all of these kinds of status things that are, I guess, in some ways, possibly working against me, in terms of me thinking I can…I would never use this phase now, but… control a classroom, I think that’s kind of what I was thinking is like, “Oh, gosh, I’m going to do this.” And so I think you’ve kind of come at it with a lot of like, I’m the authority and I know this stuff and so that’s why you should trust me to grade you. I guess that’s kind of the unspoken thing that’s being shared there. That’s not to say I was extremely strict, because I don’t think that’s true. But I do think that I felt like my job was to show that I was smart. And I can’t fault myself for that, because that is your job in grad school, to show the people looking at you that you’re smart. That is your job in college. And honestly, it’s your job in K through 12, [LAUGHTER] the way a lot of the systems are set up. I’m not saying it should be the job, but a lot of the reinforcement systems are set up, that is what you’re supposed to be doing. So I can’t get mad at myself for being reinforced doing that for the first 30 years of my life. So that’s why you have to unlearn it. Because if you’ve been reinforced and rewarded for a certain way of being, of demonstrating your expertise, and I have a right to be here, then it’s going to be hard to turn that off. And then the other thing that connects to that is when I started to really think about my values, my pedagogical values, and what I was trying to live out in my teaching, what I was trying to bring to the classroom table, I realized that like expertise and being an expert and authority that was not on the list. That was never on the list. And so if that’s true, then that’s not a value for me, then why would I be doing things that are about reinforcing authority or reinforcing my expertise? So I think that’s another thing that I had to unravel and am still unraveling,

John: When you start as a new faculty member, might that be something that is perceived as being important in terms of affecting your student evaluations, and perhaps affecting peer evaluations, who’ve also been trained in that type of perception of the sage on the stage, the scholar who’s the expert in the room, I think those incentives continue on and it’s a lot easier to break that once you get past that tenure stage.

Lindsay: I think that’s so true, and that’s one of the things I struggle with a lot in faculty development. Before I was the Associate Director for Faculty Professional Development, my kind of stair step into this position was as early career programming coordinator. And I still have that role right now, we kind of pulled that into my position. But so what that means is that in addition to doing things like new faculty orientation, I’m working really closely with brand new faculty in learning communities and book clubs, and one-on-one consultations. And I continue to struggle with wanting to tell them: “Go break the rules, like go do this. Go live out your deepest values, because I know that, at least as a person, you’ll feel better because you’ll be living aligned to your values.” But then the other side of me is like, that feels irresponsible in some systems. And so that’s why in my non-early-career-focused work, I’m working to change systems of teaching evaluations, systems of promotion and tenure and reappointment, those kinds of things… though, I mean, I think it’s a both/and… we can work on them in both ways, and recognize that it’s inherently problematic for me to encourage that. So I do spend a lot of time with both the early career folks that I work with, but also anybody is talking about get really clear about your own personal margin for error, I guess, if we want to keep going with this statistical metaphor we’ve been using… but really just like the margin for you to get in trouble, like, what realistically could happen to you if you break these spoken or unspoken rules, either at your department level, your college level, et cetera. And as long as you’re really clear about that, then you kind of know like, “Okay, how far can I push it,” and then I say push it as far in the direction of your values. And if that means, like, being radical or progressive, as far as you can go without threatening other things that are important to you. So I totally agree, John, that it’s really hard to be saying, like, go break rules and say, but that might have dire consequences for you. [LAUGHTER]

John: My advice to junior faculty depends very much on which department they’re in and the culture of that department. And I let people know that what they want to do is really good and it’s really consistent with what we have learned about effective teaching. But some of it may have to wait until they get past that tenure threshold, unless there’s some type of revolution in their departments, which isn’t always likely.

Lindsay: Yeah. And so I guess what I’m trying to do is to plant the seeds of that kind of, in a lot of cases, it is more progressive pedagogy, or just more aligned to the Scholarship of Teaching and Learning, which is not always necessarily what I would call progressive, but it’s at least aligned to something, either evidence or values. But I guess my thought is, if I kind of plant those seeds, and then some of the early career faculty can run with it, because they find themselves in a department where they can do it, the other ones who have at least heard the message, and then maybe they will circle back to that when they get to a point where they feel more safety. But I know like even in my own department, because part of my time is in a Center for Teaching and Learning, but the other part is, I am a regular faculty member, and we even had to do some of that work in our own department. I mean, so we’re psychology, we’re the people who do the science of teaching and learning at a science level. That doesn’t mean there are people in my department who do that, per se, but at least everybody who’s a psychologist, at least in grad school, learned about cognition, motivation, emotion, we learned all the things that we talk about. So we have I say, a step up in our department, even though that’s true,that we have that privilege of knowing some of that information, we still were really in that sage on the stage expectation. And like even our peer review of teaching form, if you looked at it was really a form about entertaining public speaking, I would say. Like, that’s what the behaviors that were being measured. And so we had to go through a whole process that took, I’d say, at least two years to read, design that form, and get the buy-in from everybody, senior faculty and junior faculty alike, to approve, to adopt that form. And that form is more aligned to concrete behaviors that connect to the science and every behavior is like footnoted and hyperlinked, and things like that. And that’s kind of what we needed to do to get everybody on the same page about what does teaching excellence even possibly look like and how is it different from entertaining public speaking? So I totally agree that I guess that’s another data point for our idea that context matters, context matters in that case about how progressive can you be.

Rebecca: I really appreciate the thinking through that in terms of tension or in terms of risk, and really thinking about that that context matters for each person, and that they have to determine that for themselves, and really know that for themselves, and that it’s an individual choice. And those choices might be limited by your context. And that we might not have had models that demonstrate how that might be or how we might want those values to play out in a system. So I think it’s interesting that you’re talking about having to define values, but maybe not always having a model who had those same values.

Lindsay: Yeah, that’s a good point. When I think about the things that I would most like faculty to learn from me or from other faculty developers, it’s not actually the teaching strategies. It’s really about how to be self reflective. And how do I do that? How do I figure out what my values are? How do I figure out what my risk is? And then how do I put those two things together? And I feel like that’s maybe something that’s somewhat new in faculty development, because old models were really about: “come to a workshop to learn how to do this formative assessment technique” or something like that, but that completely ignores the context and the risks and the person who we’re asking to learn to do these things. Maybe it is an “evidence-based practice,” it’s in a journal somewhere, but it would fall totally flat, given this person’s personality, or given this person’s own neuro-divergence, or whatever we want to talk about in terms of the actual instructor themselves. And so that’s kind of what I am really interested in right now is figuring out how to best scaffold people into doing that. Because then it’s not about looking at some new teaching strategy and being like, “Oh, that’s awesome” or “Oh, that’s not awesome. So I’m going to bad mouth it.” It’s more “Does that work for me? No, it doesn’t.” And that’s fine. [LAUGHTER] I decided, “Oh, I totally see why that would work for so and so, but it doesn’t work for me.” And we’re all okay with that. But I feel like we have some distance to travel, [LAUGHTER] both as instructors and as people who do faculty development,

Rebecca: one of the things that’s often associated with expertise and authority in the classroom is grading and assessment. Can you talk a little bit about where you have been unlearning in that area?

Lindsay: I just want to point out that was a beautiful segue. I love that Rebecca. Yeah. And so ungrading is something that, honestly, I’m pretty new at, but like many threshold concepts, it’s one of those things that when you learn it, it can’t be unlearned. And also, you can’t turn off thinking about it, and it shows up in everything. So that’s how it’s been for me. So I’ve only really been dabbling with ungrading the past academic year. But I have been moving towards that, I’d say for five years, probably. And so for me, it was just reading. I read the book, edited by Susan Blum on Ungrading, of course, as many people have, but it was also just reading Jesse Stommel’s posts, and all of the things that he links to there… just questioning “what are grades?” It kind of brought that kind of naive look to it. But I don’t think I recognized how ubiquitous grades were and how we just assumed that’s just part of it. That’s just what you do. And that’s funny that I hadn’t questioned that because my scholarly expertise even before coming to all of the teaching and learning was academic motivation. That’s what my PhD is in. That’s my master’s thesis… even all the way back to my honors thesis was about motivation and learning. So it’s kind of blows my mind that I had not stopped to disentangle grades before, because they’re an extrinsic reinforcer. And so maybe that’s why it was so powerful for me when I read some of these arguments as saying, “Okay, is putting a letter or a number on to an assignment, actually accomplishing important things?” And I think the answer can be yes. But just simply asking the question, that’s not something that people had done before. And I was somebody that before I did ungrading, I did a lot of feedback. So I’m well aware of work by like John Hattie, and other people that show that quality of instructor feedback is one of the things that are within our controllable factors that can move learning forward the most. It has the biggest effect sizes in learning, the quality of feedback. And so like, I knew that, and so I was always assessing work with that in mind and giving a ton of feedback, but I was ending it with putting a letter grade on it, or a number, or something like that. And so reading about that made me really question, does actually putting the letter grade as the cherry on top, does that actually add anything else to what I’m doing? And I was like, wait a minute, it’s not. And in fact, there is, again, scholarship of teaching and learning research that shows that when you give students the feedback and the letter grade, at the same time, they orient their attention to the letter grade, and sometimes never even process the feedback. And so we’ve seen all types of experimental manipulations, where if you give the students just the feedback first, and then you let a period of time pass before you unhide the letter grade or something, students actually engage with the feedback and the quality of their work improves. So if we know all that stuff, I do all that stuff before I’m grading, but I just had never sat there and said like, well, what would happen if you stopped doing that? And that is what has happened for me in the last year. And so that’s one of my most current unlearnings. And so I did it in two graduate courses. Well, one graduate course first as an experiment, because that was a 10-person cohort. And I thought, first of all, it’s really small. And second of all, they’re graduate students. And so it seemed like a much lighter lift. And then this past semester, I did it with a new grad class, it’s a statistics class. So a lot of times when people talk about ungrading, they say that really only works in writing-focused courses. A lot of people who are English professors use it. And so I was like, “Well, I’m going to do it in statistics.” And so I tried that. And then I also tried it with a 50-student undergrad course this semester.

John: Could you tell us a little bit about how you implemented this in your statistics course? Because this is something I’ve considered for my econometrics class, but I’m not really sure how I could implement it very effectively.

Lindsay: Sure. So I think it’s important to say that the statistics course is a graduate statistics course. And so I could not say that you can do exactly what I did, because it’s a different context, right? And the context is probably pretty important too. So the graduate program that I’m affiliated with is a master’s and specialist level school psychology program. So the students go through three years of training. And at the end, after they take a test and stuff, they are licensed to be K-12 school psychologists, not school counselors, that’s different, school psychologists who are doing database decision making in schools. So these are not PhD level, but not undergrads. And the other thing that it’s important to know contextually, is almost no people go into school psychology wanting to do statistics. So none of them are like, “Oh, yes, I love math. And I love statistics. I’m going to be a school psychologist.” So I’m just rare in that all those statements are true for me, but almost nobody, [LAUGHTER] almost nobody else is that true for. So, that’s important context, because I’m talking about 10 students, I see primarily women, because that is who school psychology field draws, who are math phobic, and who in their heads are thinking, “I’m never going to do stats again after this class. I’m just going to get through this. And we’re going to move on.” And so for me, the goal there is to make them realize the power of statistics in their day-to-day future career as school psychologists. And so getting really clear with myself about like, “What’s the point? What’s the goal? Who are these people?” See, notice, I haven’t said anything about a teaching technique or an assessment or anything, yet, it’s just like, what are we trying to do here? And for me, it’s them to not be afraid of statistics anymore, and to be able to use it in their day-to-day lives. And I should say the third one is for them to recognize the power of statistics for social justice and how they could, for example, in their future schools, identify disproportionality in suspensions. Meaning what if your school over suspends black children when compared to white children for the same exact conduct offense or something like that? I want them to know how to point that out using numbers. Because we do know a lot of people seem to believe numbers more than words… not saying it’s okay, but they do. So I want them to be able to use that. So because that is my focus, what that allowed for me to do is really trim down my curriculum. And so I don’t teach a lot of statistics that most people would think surely you teach this, and I’m like, I don’t, because they’re not going to use that as future school psychologists. So that’s one thing I would say is important is cut out anything you can cut out, [LAUGHTER] that’s possibly feasible, because you need the space just in your semester to be able to do this kind of stuff. The next thing that I do, and I did this before ungrading, but it connects them, is the whole class is set up to be really scaffolded. And to follow like an I do, we do, you do approach. And so what we will do is if we’re learning some specific statistic of the day, or the week, or, you know, of the two weeks, first off, give them a general overview of it, not a mathematical way, but in a problem-based way. So what if this is what a school wants to know the answer to? This is the statistic you use for that, now let’s kind of figure it out. And then we will all work through together solving it. And then I will put them into groups. I’ll give them a new data set. But it’s the same exact thing we just did, y’all go do it. And then the homework assignment is again the same exact assignment, but a new data set. And so all that’s like really intentional scaffolding. So hopefully, by the time they have to do it themselves, they totally know how to do it. Now, where does the ungrading come in? In a class like that, to me, it kind of feels like, especially with graduate students, adding letter grades on to a process that has gone from, we’re all working together, then you’re working in groups, then you’re gonna do it yourself. And there’s tons of feedback and community the whole time, it feels almost insulting to put a letter on the end of that deep cognitive work, because by the time they get to doing it themselves, they should know what they know and don’t know, they should know what they’ve mastered and what they still need help with because we’ve been doing it so much. And so it almost just feels natural to when they then turn in that individual work, if there’s something they missed the boat on, I just like, “Hey, you didn’t do this right. Here’s how to do it properly.” And then we’re going to use that skill again later down the road. To me putting a letter or a number on that doesn’t help them anymore. But the real important thing about all of this is that if we never came back and did that one statistic again, why would they ever read the feedback in the first place, because now we’re moving on to a new unit. So at the end of the course, I have some kind of culminating assignment where they pull it all together. And in order for them to be able to make the case that they have earned some certain grade at the end of the semester, one of the course objectives is the ability to use feedback appropriately. So they have to go engage with previous rounds of feedback in order to create a final product that they could use. lobby for an A for. So that’s kind of how I do it.

John: Since you do have to assign a grade at the end of the course, how do you go about that process?

Lindsay: One big thing that I have learned, not unlearned, but learned, about ungrading this year, is that the true ungrading where we never put any letters on anything until the very, very end, when our institution requires that of us, at least for me, and for my context, I think that works best with graduate students. I’ve found over the last year of doing it, that if we have a list of learning objectives, and I also have a list of skills and dispositions that we’re trying to cultivate in them as future school psychologists, if I give them that list, they are very accurate at assessing whether they’ve got an objective or they don’t. So it’s like I never have to change the graduate students’ grades unless they have been too hard on themselves. And then that kind of feels like a gift of like, “Well, this is what I see. You’ve actually mastered everything. So how could this be anything other than an A?” So, that’s good. Undergrad is slightly different, right? They are in a different place, they need more support. And also, they’re not like grad students who have all truly willingly gone on, where undergrads… obviously college isn’t compulsory, but societally it feels kind of like it is. So with the undergrads I’d say what I’m doing there is something that should better be labeled collaborative grading, not ungrading. And we started to see some discussion about this of ungrading from an equity perspective. Some students are so focused, or have been so reinforced… so we connect back to what I was saying about how you reinforce your whole life. They’re so reinforced by a system that does put letters on work, that it requires a lot of unlearning for them and it may be too much to ask within a single semester and usually within a single course. It’d be different if a whole institution was doing ungrading and that institution was set up around preparing students to be successful in ungrading. Then I think we could totally get rid of all the grades. But that’s not my context. In my 50-person class this past semester, there were two students who had experienced ungrading before, 48 who hadn’t. I was kind of excited about the two, honestly. But there were 48 who hadn’t. And so to me, it felt irresponsible to throw them in that deep end. So what we did is a whole bunch of assignments, heavy feedback, all of that. But on the more high-stakes, or slightly more summative types assessments, I did include a rubric, but it was a rubric that didn’t have points, the levels were not included, approaching expectations, meets, or exceeds expectations. And so a lot of people have talked about like kind of a two point rubric or things like that before. So that was my variation on it, just you didn’t do it, you did it but it’s not there yet, or you get it. And so I did include that feedback for them, because I felt like they needed that level of structure, but I didn’t feel like putting letters or numbers on at that point were helpful. But we did bring in letters ‘cause three times during the semester… so at the third, two thirds, in the end… they did a process reflection, which is really common to ungrading where basically the instructor scaffolds the students thinking through their body of work up to that point. Now normally in ungrading, that kind of thing happens at the very end. So they’re thinking about the body of work for the entire semester. I was thinking the cognitive load of that is going to probably be too much for my undergrad students. So let’s have them do it first, just a third of the way in, and they’re going to think through: am I meeting learning objectives? Am I’m meeting habits and dispositions? Can I give evidence for why I think that’s true? And then I have a table at the end of that process reflection that says, from my perspective, as the instructor, these are the kinds of behaviors or benchmarks or assessment types of feedback you would receive that line up with an A. These are the ones that line up with a B, so kind of self diagnose, based on all of this. And so they do that, and we’re only a month or so into the semester. And so then I give them feedback on whether or not I feel like they’re on target with that letter. And so we did that two times during the semester before the final time, which gave us the chance to get on the same page about what letters mean. But it still feels kind of like ungrading to me, because I never put a letter on a single thing that was turned in, like one assessment or one assignment. It was always assess your body of work against these learning objectives and levels of quality, assess your body of work. Next time when I use it with them. I might not call it ungrading, I might call it collaborative grading.

Rebecca: I think sometimes the use of “ungrading” when there ends up being a grade is super confusing.

Lindsay: [LAUGHTER] That’s such a good point, because now there are a couple of colleges where they truly don’t have grades, but the rest of us it’s like there is a grade at the end, y’all. [LAUGHTER] And so I think you have to have a really small cohort that you can spend so much time individually making sure everybody understands like, “Well if it were up to me, there wouldn’t be a letter at the end, but there is…” …making sure everybody internalizes that. But most people don’t teach in a tiny context like that and don’t have that luxury. So that’s why I’m kind of thinking that this kind of collaborative periodic benchmarking of your body of work so far still to me does what Jesse Stommel says ungrading is. One of his definitions is that kind of skeptical eyebrow raised at conventional grading systems. To me, collaborative grading is still the skeptical eyebrow raise. But it also is respecting the context that is our students’ reality and our own reality.

Rebecca: …really cultivates a reflective practice too. And in some ways it’s like reflective grading, reflective practice, or something, you know? [LAUGHTER] I’ve been thinking about this a lot, too. I was experimenting this last semester with such things. And I was really uncomfortable with the term ungrading when there was a grade, ultimately.

Lindsay: Yeah, and you know that bit with reflection, like we’ve known for a long time that metacognition is really strongly correlated with student achievement. And so way before I’d ever heard about ungrading or untraditional assessment and things like that, I was already doing even scholarship in teaching and learning around like, “How can I kindly force my students to reflect? How can I gently get them to actually read my darn feedback? …because I had some statistics from my LMS that showed that only 10% of my students were spending more than 30 seconds on their feedback. And these were on like really comprehensive projects. And the LMS only triggers a view of the feedback if it’s been 30 seconds or more. And I was like, it would take anybody more than 30 seconds to read the feedback, and they’re not doing it. So I’ve been experimenting for a really long time with adding on assignments where you couldn’t complete the assignment unless you read my feedback. And you’ve probably heard of these exam wrappers… is what they’re frequently called. Yeah, I got rid of exams a long, long time ago, but I still had that wrapper thing where it was like, go and tell me what one of your strengths is, according to your feedback. Like you cannot answer that thing for a grade… this is back when I did grading… unless you could read the feedback. So yeah, I think that reflection is where it’s at. I think honestly, that’s what I think this whole upgrading thing is about. it’s about two things: it’s about questioning unquestioned assumptions and assessment. And then it’s also about leveraging the power of self reflection.

John: I think for undergraduates, providing those breaks in the process of the course can allow students to do some course correction, because students tend to procrastinate, as we all do. And if they know that the final evaluation occurs at the end, there may be a tendency to put off doing that reflective practice and the course correction that might be helpful for them ,until it’s sometimes too late. So giving them that feedback that perhaps has a little bit more weight to it, or may be perceived as having more weight in terms of its impact on their overall success in the course, I think is really helpful.

Lindsay: Absolutely. And I will say one thing that I learned this past semester doing it and I added it to what I call my “to fix” document. I have one of those for every course, just a bulleted list of things like “Don’t do that again.” And one thing I added just like a few days ago, because I was turning in final grades a few days ago, was add to the rubric… so that rubric of suggested grades have these behaviors line up with As and these behaviors line up with Bs… add to the A category, “shows evidence of responding to feedback.” That wasn’t one of the things. And there were people where I was like, I’m not convinced that you listen to that three-minute recording I did about your paper because you didn’t necessarily change it. And so I’ve already been reflective myself about that should be one of the learning objectives is learning how to use feedback. And so I’m already going to be changing that in my course.

Rebecca: Leading up to our conversation you mentioned student-centered teaching is one of the places that you unlearned, too, can you tell us more about that?

Lindsay: Yeah, so this one might feel like a little bit of a left turn [LAUGHTER] because we’ve been talking about things that are I feel like usually squarely associated with student-centered teaching, thinking about the student and the instructor as a whole person, me ceding some of my authority, me doing collaborative grading like that all sounds super student centered. But the unlearning piece around student centered connects back a little bit to the risk and margin discussion we were having. But it also connects to something that we have seen is that the people who seem to be the most student centered, will sometimes kind of martyr themselves in service of that value that they hold. And so it will become clear that all of their pedagogical values are about the student. And one thing I have learned is how important it is to support faculty in selecting pedagogical values that are about the teacher as well. And this connects to Chavella Pittman and Tom Tobin’s Chronicle article and I know you all interviewed them about inclusive teaching. And it’s so funny because for a few years I’ve been doing a faculty workshop called “Inclusive Teaching Includes You Too.” And so when that came out, I was like, “What? That’s what I think too.” So, I was like, so excited. Like I immediately messaged Tom and I was like, “This is so funny when this kind of thing happens that the same idea comes out of totally separate areas.” But for so long we’ve talked about student-centered teaching and there’s always this like implicit thing that like teacher centered is bad. But I think for a lot of us, especially those of us who have been at the forefront of student-centered teaching and have continued to like “How much more student centered can I be? How much more student centered?” We’ve gotten to a place where we are thinking that the instructor is irrelevant. Like, I’ll do anything for my students, I’ll make any choices, because it’s for their learning. But I want to make sure that we don’t forget how important we are too. Because if we only do this for the students, and we don’t do this for ourselves as teachers, and really value how important we are to this whole system, then we’re going to end up in that martyr place, we’re going to end up in a place where we’re making choices that lead us to burnout, that do not respect boundaries. And so I’m starting to think about bringing us back to instructor-centered teaching, but redefining what instructor centered means. And that it doesn’t mean sage on the stage, expert on the stage. It means human in the classroom, right? It means I know who I am as an instructor, I know what I bring to the table. And that is at least as important as everything that students bring to the table.

Rebecca: Imagine that, humans in a room, all treated as humans.

Lindsay: I know it sounds so obvious when you say it, but it’s like, but we’re not living in a way that makes us think we believe that. If there were like Martians watching us, they’d be like, What are these beings doing? [LAUGHTER]

John: I think that’s especially relevant now as we’ve come through the pandemic, where there has been so much emphasis in professional development on student-centered teaching. And I think a lot of faculty are experiencing a tremendous amount of burnout, because much of that has involved a lot of additional work on their part. And that sort of balance is important and forgetting your own human needs is not going to be very helpful in the long run if you’d like to continue to be helpful for students.

Lindsay: That’s the ironic thing is that if you are too good at being student centered, you will run yourself out such that you are no longer available to be with students at all. That’s the ironic thing about it. So learning how to set these boundaries, which a lot of times does involve saying no to students, which I think is something that a lot of people think we can’t do if we’re student centered, we say yes to everything. So I think a really important thing for us to be able to do is say no to our students, which feels strange for somebody who has typically conceptualized themselves as a student-centered teacher, it feels like you’re supposed to say yes to everything the students asked for. But there’s this amazing quote from an embodiment practitioner named Prentis Hemphill. And this is what they say… they say, boundaries are the distance at which I can love you and me simultaneously. And I think teaching a lot of the times is love work, even if we don’t like to call it, that it is kind of like living out our values, living out our love for teaching and learning. And that in order to do that the best, I have to have boundaries so that I can teach you and me simultaneously. I can love, I can learn, you and me simultaneously. But I think that will be a huge transition for a lot of students-centered teachers is recognizing that boundaries are empowering, not always limiting.

Rebecca: I think that’s a perfect moment to end on. So we always wrap up by asking what’s next?

Lindsay: That’s a good question. My hope is that nothing is next. [LAUGHTER] We’re coming into summer, we’re also going through some changes on my campus that may or may not have impact on what our year will look like and things like that. And so instead of overthinking about that, I just want, in some ways, to ignore the liminal space I’m in about academic career stuff and say, “You know what, this is a great opportunity for me to not work.” And so I typically teach classes in the summer, it’s not a requirement, it’s something I do extra. This is the first time in six years I have not taken on any summer courses, I’ve taken on summer faculty development, but I’ve tried to put it all in May, or the very beginning of June. So for me, my hope is that what’s next is a lot of reading and gardening and pondering and playing with my new little puppy.

John: That sounds wonderful. [LAUGHTER]

Rebecca: That sounds like you’re putting the instructor first a little bit here.

Lindsay: Right? I mean, I am trying to live out my values and values or boundaries, joy, ease, so like that sounds like that. Yeah,absolutely.

Rebecca: That sounds perfect. Thanks for joining us, Lindsay.

Lindsay: Absolutely. It was great to talk to you all.

John: It’s great talking to you. We’ve been following you on Twitter and have appreciated all your posts and we’re glad we finally had this opportunity to talk to you and I hope we’ll talk to you again soon.

Lindsay: Absolutely.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

236. ePortfolios

As David Wiley has noted, “disposable assignments” often have small impacts on student learning. In this episode Nikki Wilson Clasby joins us to  discuss how one campus has used ePortfolios to create authentic learning experiences in their English composition courses.

Nikki is the coordinator of the English Composition Program at SUNY New Paltz.

Show Notes

Transcript

Rebecca: As David Wiley has noted, “disposable assignments” often have small impacts on student learning. In this episode we discuss how one campus has used ePortfolios to create authentic learning experiences in their English composition courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Nikki Wilson Clasby. Nikki is the coordinator of the English Composition Program at SUNY New Paltz. Welcome, Nikki.

Nikki: Thank you. It’s good to be here.

John: Thanks for joining us. Our teas today are… Nikki, are you drinking tea?

Nikki: I am certainly drinking tea, yes.

Rebecca: Woohoo!

Nikki: Would you like to know what it is?

Rebecca: Yes!

John: Yes!

Nikki: [LAUGHTER] So this is an exotic blend called Tetley, a very strong British brew, which we Brits love, unless you’re a PG Tips fan, but Tetley’s pretty up there. And I have it with 2% milk which is the best way to drink it.

John: Most of our colleagues on campus from England tend to drink Yorkshire Gold.

Nikki: Mmhmm.

John: They seem to prefer that to the other options.

Nikki: Yes, well I am a Yorkshire lass, but I have to say Tetley has that kick that I need. [LAUGHTER]

Rebecca: Good to know. I think today I have Scottish afternoon tea.

Nikki: Ooh!

John: And continuing with the theme I have an Irish breakfast tea from Twinings.

Rebecca: Oh!

Nikki: Oh very nice, that’s a good one too. I like that one.

Rebecca: This crew needs some strong stuff today.

Nikki: We need some scones now. [LAUGHTER] Then it will be complete.

John: We’re recording this at 12:30 today, and I’ve already had five meetings today including a class.

Rebecca: This is my second pot.

Nikki: I’m impressed. [LAUGHTER] You can come and have tea with me any day.

Rebecca: [LAUGHTER] Perfect. So we’ve invited you here today to discuss the use of ePortfolios in the Composition Program at SUNY New Paltz. But first can you tell us a little bit about your role at New Paltz?

Nikki: Yes, I am the Coordinator of the Composition Program. I stepped into this role two years ago, and I am also a lecturer. And so I teach mainly our upper-level writing and rhetoric courses where I specialize in visual rhetoric. And I also teach courses in what we call Practical Writing and Design which is a new course dealing with a sort of blend of graphics and writing. And I also teach a FIG, a First-year Interest Group, for the Communication Disorders. And I run practicum for our TAs.

Rebecca: So you’re not busy or anything?

Nikki: I’m not busy at all, no. Plenty of time for drinking tea. [LAUGHTER]

John: And plenty of reason to drink that tea with the caffeine.

Nikki: Which is why I drink Tetley, yes. [LAUGHTER]

John: So, we invited you here, though, because we heard about the common use of a WordPress site for the creation of student ePortfolios. And I think the first question we have to ask is… How did you possibly get agreement within a department on the use of one platform?

Nikki: [LAUGHTER] Actually it’s pretty simple, there’s no drama involved here. So in 2019 the composition committee reworked our three-credit English 180 Composition II course to a four-credit GE course, which we retitled English 170 Writing & Rhetoric. And we had been using print portfolios for a long time, and so during the process of revamping our course the composition committee reviewed how we could improve our portfolio assessment. And Matt Newcomb, who was the coordinator at the time, and I had been long advocating for ePortfolios. So during our meetings we decided that it would be a good time, seeing as we were revamping this course, to introduce ePortfolios into our curriculum. And we’d looked at options in Blackboard, but they were just too…

John: Awful? [LAUGHTER]

Nikki: Yes, too awful, but just by happenstance and pure serendipitous coincidence the university at this time decided to just opt for a CampusPress system, and they adopted Hawksites. And so it was made readily available for us to use. And so the timing was perfect, we just jumped on it right away and said, “Yes, this is what we want. This is the way we’re going to go.” So because Matt and I had been advocating for ePortfolios for so long it was pretty simple to get our program on board with the project.

John: And we should note that the Hawks are the campus mascot for SUNY New Paltz. And that Hawksites is just a campus-wide instance of WordPress, I believe.

Nikki: It is, yes. That’s exactly what it is. We just gave it the name Hawksites. Yes, it’s a campus-based university blogging website and ePortfolio tool.

Rebecca: Are students developing these in this more beginning course, and then working on the same portfolio throughout their entire curriculum?

Nikki: Well, this is what’s really interesting. So the faculty are allowed to use the ePortfolios as a tool for however they want to integrate it into their program, so they have free rein to do whatever they want with it. And we all use it in different ways and to different degrees, depending on our comfort level with technology, and how it fits into our curriculum. But as a composition program we use the ePortfolios for assessment purposes, so I can tell you a little bit about how that is organized. So whereas we can have free rein to use them however we want, we do have some very specifics that we need for our assessments. Would you like me to tell you about those?

Rebecca: Love to hear about those.

Nikki: [LAUGHTER] Okay, so for English 160 which is the basic, very first writing course, students have to go through this ePortfolio assessment at the end of the semester to determine whether or not they are fit to move on to English 170. So this is the tool that we use to make sure those students are ready for the more vigorous program. So for that 160 assessment process the students have to upload to the ePortfolio, or at least be able to visibly show on the ePortfolio, they have to have two of their strongest assignments, and they also have to have their revisions for those assignments. They can choose whichever ones they want to put on, but they have to be two major assignments. The only requirement is that they have to show that they have been able to write in different modes, different genres for different rhetorical situations. And there needs to be an element of research in their citation, you know the beginning stages of that research process. And obviously we’ll be looking for the standard of their writing as well, that’s why the revision aspect is really important. So that’s what they’re required to do for the ePortfolio. And then the 170 students, they have different requirements. But let me just backtrack just for one second. So across the board for 160 and 170 as part of the ePortfolio requirements, all the students have to create a reflective cover letter that goes up front in their ePortfolio. They write that reflective cover letter at the end of the semester and we give them questions, guidelines as to what to tackle. And what we want from them is a sort of critical overview of their progress during the course. And they have to cite examples of their writing to prove their case. So it’s a persuasive letter, and our assessors read that first, so they’ll read that reflective cover letter. And that gives us a very clear sense of what the student understands about their writing process, and that makes us feel a little bit better about whether they’re ready for 170 or not. It shows that they’re applying the techniques and skills that they’ve learned throughout the semester to that cover letter. So the 170 students, we have a very specific framework for our 170 Writing & Rhetoric program, it’s based upon a wicked question. So a wicked question might be… What should we eat? Or how do we save the world? Or what does it mean to be human in a digital landscape? And all professors can choose whichever kind of wicked question they want, that they’re excited about, and then they base all of their assignments around that wicked question. That gives us a lot of flexibility for Writing & Rhetoric, which is wonderful. So the semester is divided into two sections. We basically require two major assignments that are argument research based, and then each of those two large assignments has two smaller assignments that help students gear into those big assignments. So, for instance, you might have a proposal with an annotated bibliography that leads to a research paper. So students have to choose one of those sections. So in the ePortfolio we want to see two smaller assignments leading to a large assignment. We don’t need revisions at the stage for 170, we acknowledge that revision is part of the process, and that they will be revising anyway for those papers. So that’s the structure of the assignment sequences for those two ePortfolios. And then beyond that we add other things into the ePortfolios as we see fit. So, for instance, this semester we have our internal assessment which is for our 170 students, and that’s on basic critical reading. So that’s kind of how our ePortfolios are set up. And then at the end of the semester we have a system set up where we review each other’s ePortfolios based on a common rubric that we have put together. So that’s basically how it works.

John: It sounds like a great approach in ensuring standardization across their classes and making sure that all students meet the requirements to move on.

Nikki: It is, it’s very effective. We have lots of conversations afterwards about who’s on the cusp, borderline cases, and so it’s very democratically pieced together. And then of course we have to work on individual cases of students who are failing for various reasons. And it’s a pretty good system, and it’s been very effective over the last two years that we’ve been using it.

John: This is more of a technical question about the organization… Is each site organized by class or is it by students? In other words, does a student have their own WordPress account that they use and create an ePortfolio that is unique to them across all their classes? Or is there a class site where all the students in the class post their work? Or is it some combination of the two?

Nikki: So what happens for us is that in our composition program, we have a template on Hawksites, and our students create an account through Hawksites, and they are given the template that they have to use. So they are essentially creating their own account on Hawksites for our classes, and it’s unique to them. It’s not something that we share with other classes, this is specifically for our class. Does that make sense?

John: Mmhmm.

Rebecca: So just to clarify, if a student was in another class, in another subject area, they may have an additional site.

Nikki: Correct, students can create as many accounts as they want on Hawksites for individual programs. I have about 10. [LAUGHTER] It’s fantastic. It’s such a good resource, we love it.

John: I’m hosting a variety of WordPress sites as well for different purposes.

Rebecca: Me too. One thing I think that’s always important to ask when we’re talking about ePortfolios, is whether or not that student work is public to everyone, public just to members of the community, your classroom community? Or are they private? Or do students have a choice of the privacy settings?

Nikki: The students have a choice of privacy settings. But what we encourage students to do is to select the option that allows only people within our university that have a university ID and login to be able to access it, and only the people that the students give the link to, or the people that the faculty give the link to. And this allows us to share those ePortfolios amongst the people who are going to be assessing. So there is some choice for the students, but it also gives us the option to share easily amongst our colleagues. But I want to add something into this too, that within our template that we had created at Hawksites, we have a permissions policy embedded into the site. And that is a basic form and students can sign it, they can say yes or no. We ask the students, “Would you mind If we shared your portfolio for teaching purposes? Would you mind if we shared some of your work for research or for teaching methods?” And students can pick “yes” or “no” for all of those. And that’s nice to have that there on the ePortfolio, so whenever we’re looking for examples we can check the permissions pledge and see who’s agreed and who’s not agreed, and then of course we respect the students that haven’t signed it. So there are some levels of privacy within our cohort of teaching. There is a blog function on our Hawksite. It’s up to the professor whether they use the blog or not. But as they stand, the students can’t see each other’s ePortfolios, those are private. But there is a blog function within Hawksites, and faculty can choose whether they decide to share that blog function with other students or not. I have used that function for a different project, but I haven’t seen anybody take advantage of that because we also have Blackboard which has its own blog function too.

Rebecca: How have students responded to the idea of using ePortfolios?

Nikki: That’s a really good question. It very much depends on the instructor and how the instructor teaches the ePortfolio component. I can tell you that for TAs who are new at trying to grapple with this technology and pedagogy, some of them have in the past waited till the very end of the semester to have the students upload their work. It’s too stressful for students, they can’t handle it. It’s a lot of work to put a good portfolio together. So I make the TAs have the students sign up for an account within the first two weeks of the semester. And I encourage the TAs to find ways to get the students to engage with their ePortfolio on a low-stakes non-graded level just so that they can learn how to use all of the functions and the tools. And also get them in the habit of using their ePortfolio as a working kind of document, and not something that just gets shoved to the end of the semester. So it really depends on how it’s taught. But if you do teach it with those kinds of sensitivities in mind, and you don’t stress the students out, I find my students in particular love using their ePortfolios. They enjoy engaging with them, they enjoy seeing their work look professional on the site, they enjoy the option of using a more web-based writing process for embedding videos, hyperlinks, uploading images, embedding their beautifully designed Google slide presentations into their site. So they do enjoy that process. I give them time in class to do it so it’s very therapeutic for them. But they also appreciate learning some of the real-life skills that comes with curating an ePortfolio, and they recognize that this will help them later. So the enthusiasm for it is pretty high, and most students feel very proud of their ePortfolios by the end of the semester because they have something to show for all of their hard work, and it looks good. So they’ve adopted it really well. My worry about students is they do all this work, and they hit the submit button for grading, and then that paper disappears down the black hole never to resurface, and then they just move on. And it’s a shame because that work is good work, and we want our students to feel like they have a stake in the writing process, they have a stake in scholarship and research, and the ePortfolios provides a really nice platform for allowing them to think of themselves in that respect, and not just the humble student that struggles, if you like. [LAUGHTER] Helps them feel a little bit more professional.

John: David Wiley refers to those types of assignments that students post in Blackboard, or submit their paper at the end of the term and never see again, as disposable assignments. And having something that looks professional that they have access to, and that they can share and feel good about, is something that students really value. I’ve had students write some books in my class, and they really enjoy seeing this final product. It’s something that they can share with their friends, with their parents, with potential employers, and link to on their resumes and so forth or on LinkedIn, and they’ve appreciated that tremendously. I think you do some of the same, right, Rebecca?

Rebecca: Definitely the students love it when it’s like… It’s a real thing, a real shareable thing, with real audiences. [LAUGHTER]

Nikki: Exactly, and that’s the key thing, right? Especially in rhetoric having that real audience, it’s super, super important.

John: And in my experience it leads to a much higher quality of work when they have that non-disposable assignment. Have you seen the same?

Nikki: Yes, I totally agree. There’s a level of accountability that goes on there. So when you’re racing off an essay at the last minute and submitting it, it disappears into the black hole. With the ePortfolio it comes back to hit you in the face, and you can’t put that stuff on the web, you have to go back and revise it. And it’s really nice being in the classroom and having the students respond to your comments and make those revisions. And then you kind of hear the penny dropping, it’s like, “Oh boy, I really didn’t do this very well, I better snatch this up for the ePortfolio.” And it is very reassuring to see that in action. So yeah, it’s lovely.

John: So how have other composition faculty responded? Are they all comfortable with it? Was there any resistance?

Nikki: That’s a really good question. I know, for me, I’ve been involved in this sort of work for a long time, I came from Iowa State University here and we’ve been working with ePortfolios for a long time. And that switch, going from the paper portfolios which I hated [LAUGHTER] sorry, I hated them… Going from the paper portfolios to the ePortfolio, that’s a big mind switch to go through. So we had to work with our faculty, encourage them to set up a Hawksite of their own so they could experiment, help them feel comfortable sharing those Hawksites in the classroom so they could use that as a teaching tool. So initially there was some learning to do, and that’s great, I mean that’s great, that’s fine, perfect. So it took a while to make that switch to ePortfolios, but now that we’ve made that switch, I think we all recognize that it’s so much more accessible, it’s so much easier to organize, it’s so much easier to assess. We’ve only been doing it since the fall of 2019, but I don’t hear any complaints [LAUGHTER] about the ePortfolio. It is part of what we do now. So it’s good.

John: And that was perfectly timed to be ready for the pandemic.

Nikki: You know, it was perfectly timed for that. And what I like about it, and I think what we all agree we like about it, I encourage the faculty to have the students post the links to the faculty right from the very beginning. So that way we can just go in periodically, and we can just monitor what’s happening on there, and then we can give direct feedback to students about it. So it is, it’s a wonderful tool.

Rebecca: One of the things that you’ve mentioned is this template that you share out, and you mentioned some of the permissions that you allow students to choose. Can you talk about some of the other features of the template itself that you share with students? Like, what are some of the things built into it?

Nikki: Yes, so the template has the tabs already constructed so that students don’t have to work out how to recreate those. Obviously we teach them how to generate new tabs, but the basics are already there. So it has a homepage tab, so we encourage students to post a photograph of themselves and think about how they want to present themselves to a general audience as a student. So they have that, we work on that side of things. And then we just have the tab for the reflective cover page. Then we have the tabs for the individual assignments and their revisions,and then we have the permissions tab. And then we also include on the ePortfolio, this is a new feature, during the pandemic we had a lot of issues with attendance and accommodating students who were sick and who were in quarantine, so what we did was we posted the essentials of the course policies on the ePortfolio. And we had students acknowledge and sign that they had read them, and that they understood what those different policies were for attendance, for assignments, for what they needed to do if they were sick, all of those things we put on their ePortfolio site. So it became a quick reference guide for students that they can just pull it up, and they could see what was required of them. But also for us as faculty when students were suffering, or not keeping on track, or getting to the end of the semester and things were not looking good. We could pull that up, we could see who had signed the pledge and we could say, “Look, policy said that you needed to do this, this, and this, and you didn’t do those things.” And so that helps stem the flow of the great appeals at the end of the semester which I have to deal with. So that worked well for keeping students on track, and keeping that information transparent and clear.

John: And you have it in writing, digital writing.

Nikki: We have it in writing, and the students sign it. So it helps them take accountability for their part in this process. They can’t say, “Oh, I didn’t know about that,” when it’s on the ePortfolio and they’ve signed it. It’s like, “Mm, well apparently at some point you did read this.” So that helps. Not all students read it [LAUGHTER] of course, but at least it’s there though, that’s the important thing. Those documents are not buried somewhere else, they’re visible, they’re right up there. And I think that’s really, really important, and I’m really glad that we decided to do that, especially over the pandemic. It’s been helpful.

John: And I know I always read all the terms and conditions when I sign up for a new software package, and so forth.

Nikki: Of course we do. [LAUGHTER] The other thing that I want to add in there is that, for me, some of the professors do this too, but I have my students create a writing journal tab in their ePortfolio, and they have weekly writing journal prompts in there. And I do that so that students have a safe space just to write, and to reflect on what we’re doing in class, and to apply those ideas to material that they’re interested in. And I set that up because I wanted them to feel like they owned their ePortfolio, that it was their ePortfolio, that it was their personal sort of diary, if you like, of all of their work. So that tab is very important for my classes, and my students enjoy doing that kind of work.

Rebecca: You also mentioned earlier that the work looks professional, so I’m assuming then there’s some stylistic things that are built into the portfolio as well. There’s at least a base look for things, no?

Nikki: There is a very basic look, and I would love to be able to include more design tools in the ePortfolio because we don’t have a choice of font style, we can move our images around [LAUGHTER] to a couple of places. It’s very, very rudimentary, and it would be really lovely if we could add a few more tools in there to make it look even better.

John: So everyone in composition has agreed to use templates, but it sounds like they might use them all differently. Is that correct?

Nikki: Yes. Thankfully, even though the design elements are pretty rudimentary, there are some tools to change the actual overall look within the basic template. The students can change the background image, they can change colors, they can personalize it in a way that suits them which is really nice. So yes, that’s fun, and those are good skills to teach the students as well.

John: What about different sections of the course? Is there a standardization in terms of how the platform is used? Or does that vary from instructor to instructor to some extent?

Nikki: It varies from instructor to instructor depending on their comfortability with technology, and how they want to integrate the ePortfolio into their program. I’m not a standardization sort of person, but we do have… the basic elements for assessment are standardized, they have to have those specific elements for assessment. But apart from that they are free to use those ePortfolios as they wish, and that’s the way that I want it to be.

Rebecca: One of the things that might be helpful for listeners too, earlier you were talking about your assessment process, and that people from other sections review work, that you’re reviewing work of the students of other instructors. So I’m wondering if you’d talk a little bit about the logistics of how that actually works. Because I think for some folks it can be such a big undertaking, so hearing stories of how other people organize those sorts of things can be helpful.

Nikki: Yes, so first of all students have to be eligible for an ePortfolio review, that’s the first step. So students have to have completed all of the assignments and all of their requirements, like the library instruction, oral presentation, all of those things. The student has to have at least a D to be able to be eligible. So that sort of weeds out some of the stuff. And then what the faculty do is we take seven portfolios per class, and that’s a random selection, so you take the first student on your roster, and then every fourth student gets to go in that pile. So each faculty member has seven students randomly selected for ePortfolio assessment. Plus, we have then any student who is borderline, any student that is just clinging on there, or any student that a faculty member is really unsure about, so that goes into the pot too. And then my assistant and I, we create ePortfolio partners and we specifically place, for instance, seasoned faculty members with new TAs. And that’s the way that we do it, so we choose who assesses whose work. And that makes it a very organized system and a fair system, especially for the new TAs who are not sure about what to do, at least they’re working with someone who has experience. So that’s how we do it, and the assessments can take place whenever is convenient for that particular pair, as long as all of the results are all tabulated and submitted by a specific time period. And then after that time period we’ve got some space here to work on ePortfolios that have issues. So once that rudimentary assessment is done then anything anyone is concerned about can be given to my assistant and I, and we’ll go through case by case any of those borderline cases that we’re worried about, we can work on those. So that’s kind of how it works, and it’s a really good system. It works really, really well, it’s very efficient, it’s fair for everybody. At the end of the semester, you know we’re tired, the faculty have already been through all of the ePortfolios and given their verdict, and then we double check with those seven to make sure, it’s really a calibration thing to make sure that everybody’s on the same track. And I need to preface this by saying that all faculty members have to go through a standardized calibration training at two points during the semester, so we make sure that everybody knows how to use the rubric and can apply it effectively. So with those checks and balances it actually works out incredibly well.

John: Are there standard documents that you share with people, and then you see how they evaluated to compare against the benchmarks, for the calibration?

Nikki: Yes, we do. We have a standardized rubric, and then during our retreat sessions we will selectively pick, like, a very, very borderline portfolio for people to assess. We put people in groups, and then we make them grade the ePortfolio with the rubric, and then we discuss it and we talk about what’s working, what’s not working. And if there’s any huge discrepancies in the assessment of those ePortfolios we talk about what was going wrong with those discrepancies. So it’s pretty organized, and it’s pretty efficient.

John: It seems like a really nice way to provide equitable and fair assessments that adhere to the standards that you’re trying to meet. I’m impressed.

Nikki: Thank you.

Rebecca: You talked a little bit about students needing to meet standards to go through the portfolio assessments. Does that essentially equate to their ability to continue on in that particular program?

Nikki: Yes it does. If they’re not meeting the basic requirements for an ePortfolio review, technically it means they’ve failed. And so what we do with those students is we then decide… How did that student fail? Did they fail on their own merit? Or did they fail because they tried and tried and tried but just couldn’t get it? So we have standardized measures here that says, “Okay, so if a student has been trying really hard, and they just didn’t get it, then we will allow that student to repeat the course.” So we have checks and balances there for those students.

John: So this program is used universally in the Composition Program. Have similar practices been adopted by other departments at New Paltz?

Nikki: I’m ashamed to say I don’t know, and the reason for that is because I don’t get out much to see. [LAUGHTER]

John: That’s not uncommon especially during the pandemic.

Nikki: Yeah, I just don’t know, and I feel embarrassed to say that, but I came out of teaching a 4/4 load into this position. So that’ll be one of my next step projects is to figure out who else is using them on campus and talk to them about how they’re using those ePortfolios.

Rebecca: So another thing that is worth considering is… You mentioned that students can choose some privacy settings and things. How long do students have access to these portfolios after they’ve created them?

Nikki: As far as I know students have it for as long as they’re a student.

Rebecca: So we’ve talked about assessment being a primary motivator and maybe some professional skills as being a good motivator for putting ePortfolios in place. But are there other advantages to using student ePortfolios that we should be thinking about?

Nikki: So apart from the ePortfolios for the students being an opportunity to see themselves as professional communicators, to help boost their ethos and their confidence. I think we talked a lot about what the students get from this, but from a faculty’s perspective the ePortfolios are a fantastic tool because they are so accessible, they’re easy to coordinate for assessment, we don’t have to wade through buckets and buckets of paper. And also we don’t have to, [LAUGHTER] I know this sounds like a really minor thing, but when we used to do the paper portfolios we’d do the portfolio assessment, and then we would call the students into our offices to break the news to them whether they’ve passed or not, and give them their paper portfolio back, and a lot of students didn’t come. So we ended up with piles, and piles and piles of portfolios in our offices and it’s like, “Well what do we do with those?” I found it really distressing. You know, if you’ve got four classes and 120 students, and every semester, and then they just pile up, that was distressing. So to switch to the e-system just feels better on my soul, [LAUGHTER] for the planet doing this. But the ePortfolios, they’re just such a good tool for faculty for teaching, for training other faculty, and for sharing what we do with our students with each other, and sharing ideas and seeing what the possibilities are. The ePortfolios just offer so much more potential for pushing what writing and rhetoric is, and what we do with it in the classroom. So from a pedagogical point of view, I can’t imagine going back to paper portfolios. It’s just a fantastically amazing, creative, and soul-satisfying tool to have at your disposal.

John: That’s a really nice, positive note to end on.

Rebecca: So we always wrap up by asking, What’s next?

Nikki: Well, so last summer my colleague Rachel Rigolino and I used Hawksites to develop an online tutor training site because we need more TA tutors in our system to help with the writing program, and that was really successful. And so what we want to do now is to extend that. We would like to develop a Hawksite for our TAs so that we can put all of their innovative teaching ideas into a Hawksite, so that it’s accessible to everybody for sharing ideas. And that’s a really big project. So that’s our next big step, to do that.

Rebecca: Sounds like it’ll be really helpful, and really exciting to work on.

Nikki: I think so. I think it will be vital. [LAUGHTER]

John: This sounds like a really good program, and thank you for joining us, and thank you for sharing this with us.

Nikki: Thank you, it was a pleasure. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle, Annalyn Smith, and Joshua Vega.

[MUSIC]

218. Blended Learning

Although new to many as a result of the pandemic, blended learning has a long history of effective use. In this episode, Chuck Dziuban and Patsy Moskal join us to discuss how blended learning has been used at the University of Central Florida for the past two decades. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida [UCF] where he has been a faculty member since 1970, teaching research design and statistics. He is also the Founding Director of the university’s Faculty Center for Teaching and Learning. Patsy is the Director of Digital Learning Impact Evaluation, also at the University of Central Florida. Chuck and Patsy are both Online Learning Consortium Fellows and have been doing research on blended learning for quite a while now. They are also two of the editors of the recently released third volume of Blended Learning: Research Perspectives.

Shownotes

Transcript

John: Although new to many as a result of the pandemic, blended learning has a long history of effective use. In this episode, we examine how blended learning has been used at one institution for the past two decades.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Chuck Dziuban and Patsy Moskal. Chuck is the Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida [UCF] where he has been a faculty member since 1970, teaching research design and statistics. He is also the Founding Director of the university’s Faculty Center for Teaching and Learning. Patsy is the Director of Digital Learning Impact Evaluation, also at the University of Central Florida. Chuck and Patsy are both Online Learning Consortium Fellows and have been doing research on blended learning for quite a while now. They are also two of the editors of the recently released third volume of Blended Learning: Research Perspectives. Welcome, Chuck and Patsy.

Chuck: Well, thank you, Rebecca. We’re glad to be here.

Patsy: Yeah, thank you very much. So nice to be here talking to you guys.

John: It’s great to see you back, Chuck. And it’s great meeting you, Patsy.

Chuck: Yeah, this is my third time. I’m honored.

John: Our teas today are… Are either of you drinking tea?

Chuck: I am drinking lean green tea from a Christmas mug.

Rebecca: Perfectly in season.

Patsy: Yeah, I’m just drinking a green tea. My usual afternoon drink.

Rebecca: I think I’m accidentally drinking watered-down English afternoon tea. I think I put hot water in my cup without putting a teabag in it. And so it’s very light.

Chuck: You’ve always been innovative, Rebecca.

Rebecca: So it’s hot water. [LAUGHTER]

John: And I am drinking a peppermint spearmint tea blend, as it is late in the day and I had a lot of caffeine earlier.

Rebecca: Miss your blends there, John. Haven’t had one in a while.

John: Well, I’ve got some new fresh tea to make them.

Rebecca: I’ll have to stop over. So we’ve invited you here today to discuss blended learning. First, it might help if we start with the definition of blended learning?

Chuck: The definition of blended learning could take this and several other podcasts. The whole notion of blended learning has sort of evolved by complexity. It is diverse, it is interdependent, it is connected, and it is adaptive. I remember the early days with the Sloan Consortium where Frank Mayadas, who was adamantly opposed to blended learning, had wanted nothing to do with it. And Tony Picciano and I finally convinced him to fund a small summit in Chicago to begin discussing blended learning. And the first task, of course, was to define blended learning, and that was 30 years ago. And here we are, still defining blended learning. Blended learning is emergent, it is changing constantly. It almost defies definition. If you look at the literature, Oliver and Trigwell writing that famous article “Can ‘Blended Learning’ Be Redeemed?” and castigating it because it is not scientific, it has no specific definition and never will. To Rhona Sharpe saying its great strength is the fact that it is undefined and it allows every campus to develop its own model of blended learning. So blended learning definition is a very, very tricky kind of thing. It’s going to be some combination of online and face-to-face education, but it has evolved way past that, as you all know, in terms of doing this. I think basically—and I’ll let Patsy chime in in a minute—it’s become what Susan Leigh Star has called a “boundary object.” Boundary objects are something like critical thinking. If you go to the UCF or Oswego campus and you say to faculty, “Are you in favor of critical thinking?” “Oh yes, we are. Critical thinking is wonderful!” But then you get the group together, you get the faculty consortium together, the Faculty Senate, and they disagree. They fight vehemently over the definition of critical thinking. However, if you go back to their individual constituencies—rhetoric, physics, mathematics—they know precisely what critical thinking is. And that’s very much the way blended learning has evolved. It’s moving, it has changed the way we think about this and it is emergent. It will be emergent for the next decade.

Patsy: Just from an institutional standpoint, we do have a definition of blended learning. We actually refer to it as “mixed mode” on our campus. And it’s a combination of online and face-to-face, but it has to have some reduced seat time. So for instance, if it’s a three-credit-hour course, there may be one hour that is in a physical classroom space, with the remaining two hours being asynchronous online instruction. So, we have a newer modality that actually started in our College of Business to have large enrollment. And we don’t have classrooms large enough to fit even a fraction of the students. So, they looked at a mode that has less than 20% face-to-face. So basically, meet five times in a semester, splitting up a large class into five smaller groups, and they focused on making those active learning. And the remainder of the instruction, again, is online. So for us, it just has to have some reduced seat time capacity.

Chuck: Well, Patsy has given a great definition of logistics associated with blended learning. And then you have the educational implications of blended learning. There are great thinkers about this. Charles Graham has done a great deal of work on the kinds of blends, augmenting blends and supplemental blends and all of those kinds of things. Anders Norberg in Sweden who has done a marvelous paper on looking at blended learning simply from a time perspective… Do you want to be synchronous or asynchronous? So there are many forks to this kind of thing. What do you do with seat time? What do you do with educational philosophy? And as we talk about modality, and I will say now and people probably won’t like me, but course modality is one of the worst predictors of outcomes ever, in terms of doing this. When you look at any kind of predictive model, students don’t care about modality, we’re the ones that seem to be obsessed with it.

John: Could you tell us a little bit more about how the blended learning program at UCF got started?

Chuck: We started this whole thing with online learning. And Patsy and I began to collect data immediately. And the administration said, “Oh, this data is very informative, collect more of it.” So one of the studies we did is we looked at the presence of students who were taking online courses. And lo and behold, we found that the vast majority of them had a campus presence. All of these distance students that we imagined didn’t really exist. They were taking courses from labs, they were taking courses from the library, and they had a presence on campus. So it led to the conversation of… Well, if they’re here, can we somehow use the affordances of both modalities, of the face-to-face and the online learning? And what happened was, we developed, and Patsy’s already mentioned, this notion of, we call it “mixed modality.” We like to think that we invented it, but everyone else likes to think they invented it as well. And we stuck with mixed modality for a very long time, until Sloan began funding blended learning. And then we changed very quickly to blended learning. Patsy, what did I leave out about that?

Patsy: No, I think that’s essentially how we started doing blended learning. And we’ve never had enough classroom space, I’m sure every college is the same. So, part of the reason that we looked at online and blended was to increase access for our students, to help people, provide them with the means to be able to get an education. And it was very appealing for the administrators to look at the concept of blended learning. Again, if you took that three-credit hour and built one-credit hour with face-to-face, well I can fit three of those blended classes in one classroom over the course of the semester. So that’s an appealing use of maximizing the classroom space. And I think our past CIO, Joel Hartman’s, idea was to do that. Ironically, It’s a lot harder than you think because of the way scheduling is done. As you can imagine, departments don’t necessarily coordinate. We all know how it works functionally, sounds great, but when you try to put it into play at a university sometimes there’s things you don’t plan on. But we did have our Rosen School of Hospitality Management within the last, I’d say, five to seven years. And they’re down at our International Drive area because, not surprisingly, that’s where all the hospitality jobs are, so it makes sense. And they had a huge parking issue. So they had not been interested at all in blended learning until they realized they could use it for this exact reason. And then they jumped on the blended learning bandwagon and are now one of our biggest users of that modality. So it’s really interesting how, if you’re thinking outside the box, a lot of this comes to play.

Chuck: That’s an interesting story. There’s a great aphorism by the great philosopher, Yogi Berra, who says, “In theory, there’s no difference between theory and practice, except in practice there is.” And this is what’s happened and the fact is that although the Rosen College began their blended learning initiative out of a not-so-educational perspective, again, this logistic thing. And I will quote that our former CIO, Joel Hartman said, “If you ever think you’re going to make money with this, you’re in the wrong business. It’s not going to happen.” But the other thing has happened, the Rosen College has developed into a marvelous, marvelous educational system using the blended learning model. Maybe not for the right reason, but they have evolved,. and they’re doing things extremely well, because they’re a natural for the combination of hospitality management, face-to-face, and online learning.

John: So blended learning started there mostly because of limited resources and the need to use the resources more efficiently. How did it work? What does the research tell us about how effective blended learning has been compared to other modalities?

Chuck: Well, imagine yourself as a faculty member who has been told, voluntold, that we’re going to do a blended learning course. So now what you’re going to do is you’re going to have part of your course online, part of your course face-to-face. The first time, almost always, a faculty member starts, they start with this thinking: “What in my face-to-face can I offload to the online environment?” And they think about that, and that is usually the beginning of this. And then it begins to emerge when they say, “Ah, maybe this is not the way to think about it. Maybe I need to think about it in terms of which is most effective online, in which students do the best online, in which students excel in the face-to-face environment.” So what happens, John, is there’s an evolutionary development of faculty members, to the fact where they no longer think about this as two modalities. But they think about this as a constructed course with seamless boundaries. And that really happens over several semesters. If you’ve ever done it yourself, teaching the first course in any of these modalities, which we have, it’s a disaster. It just is not the way you think it is. We’re all pretty experienced at this, and we all know when it goes well. And we all know when it doesn’t. And nothing feels better when it goes well, and nothing feels worse than it goes badly. That’s the way it happens, it evolves. Now the outcomes from that is a story that we can talk about later.

Patsy: I think UCF… early on… we were lucky that we got involved with online and blended learning when we had enough resources to be able to do it right. And we had visionary administrators who recognize that it’s important to focus on the faculty development, it’s important to focus on research and try to use research to help inform the process. And so all of those units were put into place early on. And the late Dr. Barbara Truman actually led the Faculty Development Initiative. Dr. Steve Sorg, who’s retired, led the Center for Distributed Learning, and then Chuck led the research arm of our beginning part. So in faculty development, one of the main drivers and we’ve always said, “Don’t let the technology drive the instruction, but let your instruction drive the technology.” So, as part of the faculty development, faculty are encouraged not to just figure out what to put online versus face-to-face. But really think about redesigning your course from the ground up. What are your learning objectives? What are your instructional objectives? What do you hope to do? And then find the best tool… whether it’s online, face to face, active learning, adaptive learning, whatever… find the tools available that you can do well, and go from there. So I think that’s been part of it. And UCF also has a detailed process for quality review for courses. So if you’re teaching online or blended, you are required to go through the faculty development and get certified. But that gets you access to an instructional designer who helps you think through not just the technology, but the pedagogy behind it. You get access to graphic artists and we call them tech rangers who develop all kinds of great tools and games and simulations and all that kind of stuff. For adaptive learning, we actually have instructional designers that focus on personalized adaptive learning. So they know the software and the instructional components of adaptive learning and how to do it well. That’s the big carrot, they get incentivized to come through the program. But I think, once they get in, they actually like some of the stuff they found. So, in redesigning their course to be online and/or blended, they find tools that they then want to incorporate in their face-to-face courses as well. So that’s historically been what we’ve done, what we’ve seen. And it served us well, I think, to maintain quality across the institution.

Chuck: Well, what you see also is, that Patsy didn’t mention, is in our training of faculty, they experienced the blended learning course. They actually are involved in a course that is blended learning. So they understand, as students, what they are confronting in terms of this. So it is, again, evolutionary in terms of a changing process. And I will say that every faculty member who comes in from whatever department thinks that the pedagogy and unique problems associated with their discipline is in fact unique. That what you have to do in rhetoric, what you have to do in physics, is vastly different than any other discipline. And we are special and we are unique. At the end of this course, they realize that there is common ground, mostly all common ground in terms of what constitutes effective instructional things. And they’re able then to look at all of these technologies we present to them and be able to evaluate them, “Is this technology really a solution looking for a problem, or is it something that can actually help me become more effective in my instructional design?” And it is, again, an evolutionary process. And you’ll see a coming together of faculty during this experience. And they don’t all buy it. We’re instructionally design oriented. And I always remember an incident of someone from rhetoric came into my office and threw a copy of Derrida on my desk and said, “What is this instructional design stuff? When you understand deconstructionism, we will talk.” So I read the book and said, “The book deconstructed and disappeared. Now, what do you want to talk about in terms of doing this?” But the point is, you have to respect the values of the discipline. And I say that with tongue-in-cheek, but you absolutely have to respect the value structure of any discipline you’re working with. If you tell them they’re doing it wrong, you’re not going to get far.

Patsy: We also rely on something called “web veterans.” We have our web vets, which are faculty who have already been through the process. And not surprisingly, I think anytime you have a new initiative, there are those people who jump on board, the first people out of the gate who love to test drive stuff. But they become the experts, and they know how to do it well, and they know the pros and cons. And they also add credibility. So faculty listen to other faculty, particularly those within their discipline, or STEM listen to other STEM faculty and so forth. But it’s much more credible than having an instructional designer telling me how to teach when you’re not in my classroom, as Chuck said. So I think it’s really helped. We have a faculty advisory board that really helps direct what we do and think about the evolution as we’re now in a state of evolution, I think, and with COVID, are going to be in a state of evolution. I think that really has helped.

Chuck: Well, we’ve all taught, we’ve all taught for a long time. And there’s a great deal of difference between teaching a few courses, and teaching year after year after year after year over the long haul of this, and we’ve all taught and we’ve all taught semesters. And I think all of us experience this excitement, and anxiety at the beginning of a semester. You say, “I really want to do a good job, and I’m all excited and I’m nervous.” And at the middle of this semester you say, “My god, will this thing ever end?” You’re trying to get through it and trying to work through this sort of thing, and at the end of it you feel a sense of completion. And that rhythm is so important to understanding the instructional process, not only with technology, but as a faculty member in general. And we have to respect that teaching a long period of time is a difficult task. And you all know that classes are very much different. What goes well one semester does not go well the other semester. We’ve all experienced that, we’ve all experienced that.

Rebecca: I’m living that.

Chuck: Absolutely, and it’s draining, Rebecca. It is draining because you contribute a great deal of emotional intellectual energy to this task. We all do. We were talking about boundary objects. Equality is a beaut, it is a beaut.

Rebecca: You’re talking a lot about some really important resources that are in place at your institution: a pretty expansive support system of technology support, instructional design support, actual technology [LAUGHTER], peer support, time, professional development. That leads me to think that perhaps one of the biggest mistakes people make, or institutions make around blended learning, is not putting enough resources into the system?

Chuck: Yeah. But again, I think you have to be careful with, “If I had more money, I’d do a better job.” The whole notion is not only that, but commitment at the administrative support level, and all particular levels, and basically saying, “We value the teaching enterprise, and we’re going to do everything we can to celebrate that.” That is critically important, because money alone doesn’t do it. You know that as well. I had a friend once who said, “If you had all the money and everything you wanted to do your job, two weeks later you’d come back and say, ‘I don’t feel fulfilled.’” Because it takes much more than that.

Patsy: Well, and it’s easy to hear us talk about all that we have in place now. But understand, we started in 1996, so we didn’t start with all of this. We started with instructional designers with TV trays and laptops, meeting people at the campus coffee shop or in their offices. And it started small and grew as the initiative and the institution grew. So over time, it’s easy to add resources. But I think one of the common mistakes is just underestimating the amount of work it takes to produce a quality online or quality blended course. And not meaning to be a Debbie Downer, but segueing into what we’ve experienced during COVID, we’re very careful to make sure to differentiate what we had to do from an emergency remote instruction, and say, you know, “That is not the quality online learning that we have worked so hard to put into place for the last few decades.” So I think that’s very different, in terms of just trying to survive and keep the doors open, so to speak, for students, with everyone… those who have never been through any training, those who’ve never had any online-teaching experience in the middle of a pandemic with kids at home and scant technology resources… students who don’t have any technology resources, who have major job issues, family issues, in the least-conducive-to-studying room and try to expect everyone to excel. I think that’s really the fact that it does take a lot of work. And so I think that maintaining realistic expectations, as Chuck said, it’s not a one-and-done kind of thing. It’s not a one-and-done when you teach face-to-face either, right? The first time you teach any class, it’s not the best time you teach. You evolve, you learn, you adjust, and continue to try to, hopefully, improve every time you teach. I think that’s true of any of the online or blended modalities as well.

Chuck: Steven Johnson has written a wonderful book called Where Good Ideas Come From. It is just a really, really good book and he has three adages in terms of this. One is… What is the adjacent possible when you start this? What is the reasonable next thing you can do? You can’t do it all, so do the thing that you can do and do it well. That’s the adjacent possible, it comes from a biologist named Stuart. The second thing is a slow hunch. You have to be in it for the long haul. If you look at Darwin, he was in it for the very long haul. And he didn’t even know what he had for several years until he came up with it. You have to commit to the long haul to do it, not a quick fix. None of this is a quick fix. And the next thing is a liquid network. You need to be supported by a network around you to prop you up as you go along. And Rebecca, when you said this in terms of resources, one of the resources is you are in this for the long haul if you’re going to do this. So it takes an institutional commitment, and that always is not there.

Rebecca: What are some of the most promising areas of development in blended and online learning?

Patsy: We’ve had good success using adaptive learning, we have an adaptive learning initiative. We started in 2014, I guess. And again, we learned it takes a lot of work, we use a content-agnostic platform, so Realizeit is our enterprise solution. We do have, our math department uses ALEKS as well. But it takes a lot of work. Faculty want control, and you get control, you have to redesign the course. But you have to actually design all of the content for a lot of it, or at least the majority. And trying to import from another sounds great, but it always still requires work to get it to work well. Though, that’s one of the things that we’ve had good luck with. I think we’re going to see a lot more blended learning post-COVID. We’ve never really promoted synchronous online learning and haven’t seen a big drive to do synchronous until everyone had to do Zoom. And now I think there’s some faculty that are going to run for the hills as soon as they can get out of Zoom mode and never want to look back. But there are some who were already asking, “You’re not going to get rid of Zoom, are you? Because I found it’s really great for me to do these discussions…” So there are people who’ve already, again, figured out instructionally how they can use this effectively, and do it from a distance. And when you’ve got students who might be geographically dispersed or have to worry about traffic, we have parking issues, all of that stuff comes into play. So I think we’re going to see a lot more blended learning in general in the future. And not just in instruction, but also in the workplace. We’ve seen pilot testing in a hybrid work environment now with so many days in the office, so many days remotely. So, I think that’s one thing.

John: If you’d like to learn more about Realizeit and adaptive learning at UCF, we did a podcast with Chuck a while back, episode 30, on adaptive learning, and we’ll include a link to that in the show notes.

Chuck: Oh my God, way back to episode 30. We are dinosaurs, aren’t we?

John: Since a few of our listeners might not have been listening back in the primeval times of episode 30, could you briefly explain why adaptive learning might be useful in supporting students with diverse educational backgrounds?

Chuck: I think you really have to ask yourself, “What is the problem we’re trying to solve in the United States of America?” I’ll get it from a different point of view. I think a problem in the United States is that if you live in a lower economic quartile, your chance of going to and graduating from college is 10%. The odds against you are nine to one. I think blended learning and adaptive learning and some of the things that we’ve learned during this pandemic, are going to help us a long way to begin to solve that kind of a problem. That is, we know that many of our underserved students live in an environment of scarcity. They don’t have enough resources, they don’t have enough time, they don’t have enough transportation, they don’t have enough support from their families. They have simply no opportunity to engage in the university the way we’ve constructed it in the years past. They can’t do it. So we also learned, and very much from several papers and the notion that, if you fix the amount of time that a student has to learn, what they are going to learn is going to become the variable. If what they learn is constant, then the amount of time they spend learning, surprise, is going to be the variable. We’ve confronted this, it’s a difficult challenge for universities. But I think one of the real affordances of what we’ve got is to begin to look at… How can we, as universities, eliminate this horrible educational inequity, which exists in the United States of America? That’s a real affordance that we’re going to learn. John Carroll’s great paper, “A Model of School Learning,” said 60 years ago that we have to find another way, that the students do not start from an even playing ground. And you take an underserved student who comes in and one domino falls, their car breaks, or they can’t get childcare, the entire system collapses on them. They miss one class and they’re behind, they miss two and they’re lost. The optimal choice is to drop out. That is not the choice that we want. And I think the things that we’ve talked about today are going to really help us address these very serious problems in our country. You know it, we are wasting millions of minds in the United States, and we can’t do it anymore. I don’t know if blended learning and adaptive learning will solve it all, but it’s going to go a long way to help, I think.

John: And that issue of income inequality and the growth of that is really troubling. We bring so many students into our campuses, and then we just let them fail out without providing support so that they can be successful. And it’s a waste of resources, and those students are ending up burdened with a lot of debt once they leave, without getting the benefits of that education. And I agree, this is one of the most critical things we need to work at.

Chuck: Let me address the debt problem, too. That’s a huge problem, John. Right now, the accumulated college debt in the United States is $1.7 trillion. That is mind blowing. If that were a GDP, it would be the 13th largest economy in the world. We have got to do something about this in this country, we have got to find a way. And like you just said, tragically, if you drop out, you’re in debt, and you’re in debt for nothing. And Pell Grants are wonderful, but Pell Grants are two semesters. One semester, you get on probation, you have one more to get off probation. If you don’t get off probation, you’re no longer Pell eligible. Do you know how hard it is to bring up a whole semester of Ds in one semester? It can’t be done, can’t be done.

John: One of the things you mentioned is that blended learning is likely to be adopted more as a modality moving forward as colleges have experimented with a wide variety of modalities. But for many campuses, this may be the first time they’re doing it. What are some of the most common mistakes that faculty make when they build that first class? You mentioned before, Chuck, that these classes, the first time you teach them are often unsuccessful. What are some of the things faculty should avoid doing when they build that first blended learning class?

Chuck: I mentioned earlier that when they begin, they view this as two separate modalities and what of my face-to-face… what of the gold standard can I offload? And the thinking of, “I’ll offload the unimportant stuff to the online environment.” It’s wrongheaded thinking. In terms of I think what they have to think about really is, “Are there students in my class who can learn better online? Are there students who function better in the face-to-face environment? And can I somehow encompass that?” And it’s all the things that we wrestle with all of the time in all of the courses. In all of my courses that I’ve taught, I wish I did everything well. Sadly, I do not. I wish I were excellent at everything. And some things, I just cover material. And I think faculty have to come to terms with, “When am I covering materials?” It’s a very introspective kind of thing. And I think you’re not necessarily ready for it the first time because, as Patsy said, I think the first time you’re in survival mode. I think Patsy will agree that the first time you do you say, “What have I got myself into, and why am I doing this?” It is evolutionary. That is a mistake to think about that. And I think a mistake is to think it’s going to go well the first time, it hardly ever goes well the first time. It’s like the first time we ever taught a course. I didn’t know anything the first time I left Wisconsin and came to then-FTU, I didn’t know anything about teaching. So I lectured for three hours. My God, I don’t understand why they didn’t lynch me. And I didn’t know any better!

Patsy: I think Chuck’s right, it’s a lot of work. And so from the faculty perspective, I think trying to have realistic expectations and not look at it as, “Here’s the endpoint of where I want to go, but you have two weeks to do it.” That’s not going to work. Make sure you have enough time. If you have access to resources, I think that’s really important.From an institutional standpoint, we know it takes a lot of work to create an online class. Faculty would say it takes more work to create a good blended class. So you can either have the best of both worlds, but it could also be the worst of both worlds. And you don’t want that to be the outcome. So trying to have realistic expectations and then, from an institutional standpoint, make sure you have the support that you need, not just for faculty, but for students. Make sure that you have the robust infrastructure. And for us, what we found, especially like this last year with COVID, is that we did have a lot of students who were doing those courses, we have a very small number of fully online students that take only online. What our students do is really pepper their courses with a mixture of modalities. So they take an online and a blended and a face-to-face. And we’d like to think that the online and blended help reduce the opportunity cost for getting an education. So they can help them balance their life with education. But we have a lot of labs on campus. They were relying on the computers in the labs, they were relying on the Wi-Fi in their dorms, the laptops that they might have access to, and things like that. So I think just making sure that you have the infrastructure that’s necessary, that you have support to be able to handle any issues, questions, both for students and for faculty. From an institutional standpoint, I think it’s really critical. And if you’re going it alone and you’re a faculty member, and there are people who do that, understandably, if you’re in a situation, start small. Keep your expectations realistic, and grow your course as you learn, as you go along. So keep improving it.

Chuck: One of the strategies… UCF is a very selfish place. In all of these kinds of initiatives, what we do is we cherry-pick the best faculty we can find to maximize the effectiveness of what they’re doing. When we did our adaptive learning things, we had faculty members who looked and said, “Oh yeah, I want to do this.” And we knew they were the best faculty we had and we knew they would make it look good. One colleague in nursing, who has left us by now, she said, “I don’t want any of your tutorials. I just want to mess around with this, RealizeIt, and figure it out for myself.” And she did, and she was marvelous. And that kind of a model really motivates faculty. I would point out so far we have not talked one iota about blended learning. We have talked about blended teaching. That’s what we talk about. We call it blended learning, but we segue, to use Patsy’s word, right away to blended teaching. And that’s an evolutionary thing, Rebecca, that we’re going to have to deal with in the years to come in terms of what is blended learning, aside from the arrangements for teaching. And the other thing that has happened is, you can blend many things. We’ve talked about blending courses. You can blend a university, you can blend all kinds of aspects of this institution in terms of faculty, and students taking online and blended and face-to-face courses. Blending their locations, blending many, many things that people have written about. Our dear departed colleague, Karen Swan wrote about the blended university and several others have talked about that notion of expanding it behind the notion of just a course, that it is much more than that. That is blending a university culture.

Rebecca: Chuck, I’m glad that you brought up students because one of my next questions was going to be: How do we prepare students for this kind of learning? Sometimes they come into an environment. And I know we had a lot of students say this in the emergency teaching that we were doing, like, “I don’t know how to learn online, I don’t know how to do this. I don’t know how to balance my time, I don’t know how to manage different modalities and things.” So what are some things that we can do to support actual student learning in an environment that might be unfamiliar to them.

Patsy: As soon as you said that I gravitated towards the faculty I work with who teach predominantly undergraduates and a lot of freshmen. Now, I don’t want to say they parent them into success as much as they can. But in some sense, that’s what they do. I think it’s probably idealistic for us to assume that freshmen come in and are going to go from being handheld with what we do in K12, to instantaneously going to be able to jump into a course and know what they’re doing and take responsibility and all of that kind of stuff. So faculty are very strategic at using the technology to nudge them, remind them, try to provide some motivation for them to get involved. And I think that’s some of what is important, I think that that helps students ease into this. We find a lot of… maybe it’s because of Florida… but I think Florida requires our high school students to have some online learning experience. But even then, the high school online learning is not the same as what we do. Because again, there’s parent consultations, and there’s a lot more very structured, dedicated time with parents and students that you don’t get, I mean, we just can’t do in higher education. But I do think there’s some strategies. We are seeing a lot of colleges are trying to work to find out who is at risk earlier and hopefully intervene. So what interventions do we have? That really early identification, trying to identify students who are in trouble and making sure they get access to the help that they need early on is important. Some of the analytics work we’re finding, you’ll see, I think, a trend that’s going to continue, as we have more and more access to student engagement, student performance with both platforms and our LMS and all of that. Everybody’s shooting out analytics, right? I mean, we know we track people all the time now in terms of how they’re doing, where they are, and what’s going on. Some of that is helpful, it’s going to be important to sift through the noise to find what’s really helpful for those students who need help.

Chuck: Well, Rebecca, I would say one is you certainly cannot throw these students into the breach. That is the first thing you cannot do, absolutely. In terms of, “You are on your own, and you’re going to do this.” So what you have to do then is really be very introspective in terms of, “What is it I do well in my face-to-face? And how can I translate that into an online environment?” And you know very well, I would rather give a lecture to 5000 hostile politicians than to do a webcast, simply because I am not nearly as effective in that environment. So that’s something that you have to do. And the other thing I think you have to realize is that the student voice has become increasingly important in higher education. We came from a time— at least I did—when you went and you lectured and you left, and it didn’t really matter what students thought. That is no longer the case. They have a voice and they express it continually about the quality of education and they share it. And you must realize that culture is going on all the time. All of those things are coming into play that change the educational environment and blended learning. Clay Shirky’s got a great thing that he did several years ago about… we used to lecture and that was it. But then they wanted to email us and they became really annoying. They wanted access to us more than just our office hours. That was a big change for many of us. And then they began talking to each other about us. And then they began talking to each other more about us. And then they began talking to the world about us. And I’ve used this example all the time, just go on YouTube and pick any topic related to teaching, and a professor, a good professor, bad professor, drunk professor, stoned professor, and you will find videos. Students now share their voice, and they’re part of the voice. And I think one of the big problems that we have to face now is: How are we going to integrate the new, more powerful student voice into the higher education culture? And I don’t think we’ve even begun to address that. And I’ll tell you right now, it is not with student ratings. This is another opportunity, as my administrators say to me, “I have an opportunity for you.”

John: So as we’ve talked about before, last year there was that whole experiment with remote synchronous instruction, where both faculty and students were thrust into new modalities that they were not used to. And there’s a lot of evidence of some significant learning losses, and a change in learning practices that has led to some challenges facing faculty and students as we move back to whatever this new normal happens to be in the middle of a pandemic. How can we address the larger variance in prior learning that students bring into our classes no matter what modality they’re in?

Chuck: Well, what we just experienced is something we knew existed all of the time. What this new experience has done with this very nasty virus is it simply exacerbated those differences. And we are going to have to spend some time, I think, devoting our institutions to how we can recover students. Because clearly, in my mind’s eye, the most vulnerable among our students are the worst impacted by what has happened in the last year and a half. Yet again, John, it is not evenly distributed. Unfortunately, it is unfairly distributed. There’s a great book that I just read, called The Class Ceiling. There’s a distinct wealth advantage in this country, people who are exposed to resources and stuff did much better in this environment than people who are not. And we’re going to have to spend some time trying to recover from that. And frankly, I don’t know how at the moment, I really don’t know how. I hope it hasn’t been too disastrous for us, but it has not been a good scene. As Patsy has said, people have tended to conflate what we did in the last two years with online learning, and they shouldn’t be doing that, but they do it. You know it, you know it, and I know it.

Patsy: We’re also seeing enrollment starting to drop. And I think one of the things that’s going to be really important for institutions to do is keep track of: Who are we losing if our enrollment goes down? Because I suspect it’s going to be some of those people who are our underserved population to really are now, through impact of COVID, forced to do something other than go to higher education. Either they can’t afford to do this, or they have to work full-time more than before, care for family, something… but it’ll be really interesting to see. I don’t think we’re quite there yet. We’re starting to see trends. And I think I’ve heard that across multiple institutions in terms of enrollments. We knew enrollment was going down prior to COVID. But now we’re starting to see some of it is maybe more than we expected. So I agree with you, John, I think it’s going to be important for us to really track that and figure out what’s happening and how we can adjust our instruction and whatever we have to do to bolster and help students succeed, whether it’s due to our instruction, or just no fault of their own due to last year. I think we need to try to figure out how to do that.

Rebecca: There’s a lot of shifting expectations that have happened as a result of the pandemic, in addition to, you know, “What is online learning, right?” And what people have experienced as emergency teaching, but also just other expectations around time commitments, flexibility, taking care of mental health, like all kinds of expectations have just shifted wildly from [LAUGHTER] maybe the way that they were before from both the student perspective and the faculty perspective and the institutional perspective. So I think there’s a lot of shifting that’s going to continue to go on as we all try to adapt to what’s going on around us.

Patsy: Absolutely, I agree with that. I think a lot of human resources departments are going to have to figure out how they need to adjust [LAUGHTER] some of their long-set-in-stone policies because we are evolving. I’m not sure we know where we’re evolving to yet, not there. So we’ll find out.

Chuck: Rebecca, I think you’ve hit on the problem of the century, in terms of… How are we going to re-examine our institution in terms of what our value structure is, what we have been, and what we need to be in the future? I’ve read a wonderful book now called Subtract, where the point is made that every time we try to improve, we add something. We never think about jettisoning anything that we have done, and it may be time for us to begin to consider what are some things that we have been doing that are no longer effective and they need to go. It may be painful, but I I think it’s a lesson we’re going to have to address. It’s just a marvelous notion, you just never think about dumping something. We’re going to add in more… give us more resources, give us more faculty, give us more this, give us more that and we’ll do a better job. Will you? I don’t know, I don’t know.

John: One of the things that I’ve been really impressed with is all the research you’ve been doing on blended learning. And one outcome of some of that research, as well as the work of other people, has been that series of books on Blended Learning: Research Perspectives. Could you tell us a little bit about how this collection of research came about?

Chuck: Well, since I’m the old guy here I can tell you exactly how it came about. Frank Mayadas allowed us to fund three summits at the University of Illinois, Chicago, run by Mary Niemiec, dear colleague of mine. 30 of us got together and discussed this notion of blended learning. And what we did is we discussed, in three groups of ten, three topics: quality—God forbid, I’m glad I wasn’t in that one—and the second one, logistics, and the third one, research. And Tony Picciano and I were in the one: research. And we were in a two-day meeting and finally Tony leaned to me and said, “This is not going anywhere. Where are we going to go with this, in terms of research?” Then the next morning, he got up and said, “We need to do a book.” So the first thing we did is we entered the meeting, he said, “We’re going to do a book,” and we got those people who are interested in doing a book. And then, at that time Sloan was running a conference on blended learning, and so Tony and I went and made a pitch about this initial book that Sloan published in terms of blended learning. And there was tremendous interest in it, and all kinds of people submitted chapters. So we had a book that Sloan published. There was a book and it went out. And then it was successful. In this niche market, a book of 1000 is successful, it’s not like Shades of Grey. And then we got to thinking with Charles Graham, ‘How about doing another one?’ And we went back to Routledge and they said, “Yeah, we see that that book was successful. So we’ll do another one.” And we did the second in the series for them. And then thirdly, they came to ask, they said, “The book did so well…” whatever that meant “…we’d like you to do a third version.” And that’s the motivation for the third book. We got a trilogy on this because Frank Mayadas didn’t like blended learning. But he’s a convert now, he’s come around in doing this. And the last book, we really jacked up the quality because we added Patsy to doing this. And now we’re moving on to a book on analytics and adaptive learning. I’ve got to stop writing books. That’s how it happened. It happened by accident.

Rebecca: Many great things do.

Chuck: Oh, they do.

John: Are there any other things you’d like to add?

Chuck: I’d like to address the fact that I think how valuable your podcasts have been for education and the nation. And I think we’re talking about a classic example of what we’re talking about. Look what’s happened over the last years: the value and importance of doing podcasts. Obviously, yours is one of the premier, and I’m not stroking you, you know I don’t need to stroke you. But the notion of this is another modality for learning that we never really considered, right? And this is what we have to do. What are these things? Cherry pick the things that are really working.

Rebecca: So we always ask and wrap up by asking, “What’s next?”

Chuck: Patsy, what’s next for you? Cause I know what’s next for me.

Patsy: Well, yeah, what’s next… wow… is surviving COVID. That’s number one. I think that’s our top priority right now, making it out of the pandemic and figuring out where we’re going to evolve to, as we’ve already talked about. Chuck and I have this book that we’re just now working on, which is data analytics and adaptive learning. So that’s coming down the pipe. And I don’t know, for me, it’s like trying to figure out how to continue to help facilitate research and enjoy what I’m doing, trying to make a difference with as many students and as many faculty as we can. So yeah. That and counting down the days to retirement because I know that’s what Chuck’s going to say. But I have a lot longer, I think, on the track than you Chuck, right?

Chuck: This is my 52nd year at UCF, formerly FTU. I think what’s going to be in the future, for me, will be retirement, but it will also be trying to reflect on what we have learned in this educational enterprise and what we need to learn as we go forward with this. Personally, it is going to be, John, and you know about this, I will be moving over to working with our underserved communities to try to solve the problems of inequity that is so crippling our educational system, and we need to work on that. But those are the kinds of things. I guess I would say thank you for having us again, and we look forward to hearing what we had to say. Right, patty?

Patsy: Yeah, it’s been a lot of fun.

Rebecca: Thank you both.

Chuck: You guys are great.

John: One thing I do want to mention is that we also had a podcast with Chuck, where he did talk about some of the ways of working with this in Episode 115, where Chuck and Harris Rosen talked about the Tangelo Park project.

Chuck: Just as an aside, the Travel and Leisure Co. has adopted another community based on what we’ve done, John, in the Eatonville community. And we’re very close to a foundation in the Midwest doing the same thing. So we are making progress. Hard as we try to get people to replicate it. I think we’re at four now.

Rebecca: That’s great!

Chuck: Yeah, it is great. It is great. And it really does work.

John: Well, thank you. It’s great talking to you always. And we’re looking forward to sharing this with our listeners.

Chuck: We really appreciate your thinking of us. We really do. Go Lakers!

John: Go Knights!

Chuck: Take care, everybody.

John: Thank you.

Chuck: Thank you so much. Bye bye.

Patsy: Thanks, guys.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

217. Grading Justice

Traditional grading systems can encourage students to focus on their grades rather than on their learning, and favor continuing generation students who are more familiar with the hidden curriculum of higher ed. In this episode, Kristen Blinne joins us to discuss grading strategies that promote equity and encourage learning.

Kristen is an Associate Professor and Chair of the Communications and Media Department at the State University of New York at Oneonta. Kristen is also the editor of Grading Justice: Teacher Activist Approaches to Assessment. Judie Littlejohn, the Instructional Designer at Genesee Community College and a frequent guest on the podcast, joins us again as a guest host.

Shownotes

Transcript

John: Traditional grading systems can encourage students to focus on their grades rather than on their learning, and favor continuing generation students who are more familiar with the hidden curriculum of higher ed. In this episode, we discuss grading strategies that promote equity and encourage learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Kristen Blinne. Kristen is an Associate Professor and Chair of the Communication and Media Department at the State University of New York at Oneonta. Kristen is also the editor of Grading Justice: Teacher Activist Approaches to Assessment. Judie Littlejohn, the Instructional Designer at Genesee Community College and a frequent guest on the podcast, is joining us again as a guest host. Welcome, Kristen.

Kristen: Good morning. I’m very happy to be here with both of you.

Judie: Good morning.

John: Today’s teas are… Are you drinking tea?

Kristen: I am, it is a jasmine green tea, my favorite of the tea world.

Judie: And mine is a Twinings Lady Grey.

John: And I am drinking Tea Forté black currant tea, which is one of my favorites, with some honey from Saratoga Tea & Honey. I think you both met through Judie’s work on the FACT2 Subcommittee on Social Justice Assessment. Could you tell us a little bit about that?

Judie: The subcommittee is part of the larger Innovations in Assessment Committee and Chilton Reynolds is one of the co-chairs of that, and he is at Oneonta where Kristen is. And so he notified our group that she had published this book. And Chris Price in our group reached out to Kristen and asked if she’d be willing to meet with us. And she did, and we had a great conversation. Unfortunately for me, I was driving that day. So I was on the thruway trying to participate as well as I could. And it was great, it was a very engaging conversation, a nd it was great to hear from Kristen and hear her enthusiasm and all her great ideas. So it’s great for me to meet with you again and to finally see you this time. [LAUGHTER] So welcome, I’m glad to see you here.

Kristen: Thank you, I was so excited to receive that message about the FACT2 Social Justice Assessment group. I didn’t even realize that that existed at the time that that email was received. And so I was truly overjoyed that these conversations were happening in a coordinated way in the system, and to be able to just have that moment to jump into the conversation here, kind of what the group is doing and the vision for moving forward. And it’s exciting to see even the website materials that are up that are situating these conversations for a broader audience.

John: We’ll share a link to the website for that group in the show notes. So, we’ve invited you here today to talk about this book. Could you tell us a little bit about how this project came about?

Kristen: Yeah, so the Grading Justice book project really grew out of a few different streams in my experience just teaching, but also my experience as a learner, as a student going through a system. I have to say that it was born out of a lot of frustration that I had as an instructor. [LAUGHTER] And I hate to frame it in that way, because it’s kind of a negative framing. But I think most instructors can agree that we spend a lot of time grading, and the grading comes with a bunch of challenges, especially if you’re doing any kind of activist or social justice work or you’re trying to create equitable learning spaces. Trying to do grading in a fair way that best meets the goals of the course and to have students on board with that process is difficult work. And so in my case, I looked back to my experience as a student when I tried to build my classes coming in as a college instructor. And I realized that as a student, I really didn’t have a very positive relationship with grades. They didn’t matter to me that much, I was always in it for the learning, but I had a lot of difficulty going through the system because of the emphasis on grading. I wanted to be a learner and I wanted to explore my creativity in ways that maybe didn’t fit with the limited grading structures that I encountered as I went through K through 12 and into my college experience. It wasn’t until I finally landed at Goddard College in Plainfield, Vermont, an institution that doesn’t do grading, that I’ve really embraced my learning in a new way. So when I came into my own teaching process, I realized it’s like, “Wow, okay, I have to now impose these grades on the learners in my classes. And how can I do that in a way that honors the journey I went on, recognizing that other students may be also experiencing the same struggles that I had?” So I came to this Grading Justice project because I wanted to have these conversations with educators, really across disciplines, but the book is focusing a lot on people who are working within the field of communication and critical pedagogy but also critical communication pedagogy. But my hope is that it has appeal to a wider audience than that. Aside from the frustrations, another thing that I noticed because I do a lot of social justice work is… How do you assess social justice work in an equitable way? So that was one of the questions I came into the project with: How do we assess social justice work? But then… What would a social justice approach to assessment look like in our teaching and learning processes, especially in the realm of critical pedagogy, because it was part of a critical pedagogy book series? I’ve noticed that a lot of people who embrace critical pedagogy, they may still be using traditional learning systems. And so how do you, as an educator, work with that tension that might be coming with reinforcing systems that may be perpetuating inequality while you’re also trying to undo systems of oppression and engage in power sharing with students? And so I was finding this contradiction that inspired me to want to really pursue this in a very serious way.

Judie: A major theme of the book is that grading systems can either perpetuate inequality or work toward equity and justice. How might traditional grading and assessment systems perpetuate inequality?

Kristen: So one of the things that really spoke to me as someone who was trying to navigate how to create a more equitable classroom, because I’m really invested in making my classroom as diverse and inclusive and accessible as possible, is having conversations with students about assessment. Because what I found over time is that students, as I mentioned about myself, many of them have a pretty negative relationship with grades and grading. And a lot of them don’t see them as accurately measuring their learning, at least in the experiences I’ve had in having these conversations across my classes. I had this lightbulb moment when we had a guest speaker come to SUNY Oneonta, Ernest Morrell came as part of one of our teaching institutes. And he said something in a small group session that, really, I think about a lot in regard to this question about grades perpetuating inequality. He said, “We need to stop measuring students’ success against failure.” And in that moment, when those words kind of tumbled out of his mouth, I just was like, [EXPLOSION SOUND]. It occurred to me, I always have wanted to encourage… I don’t want to say encourage failure in the class… but I want to encourage experimentation… classroom as living laboratory where people can try things and have it maybe not work out. An example I could give of that is I taught public speaking for many, many years. It’s a class that students have a lot of anxiety about taking in many cases, and there’s a struggle there. And so one of the things one of my students did once is they decided to sing a poem about Rachael Ray. I wasn’t sure if this was a great idea, but they did it. And if you ask them, they would say it was an utter failure. And it didn’t work out in the way they wanted it to. But why would I grade someone on that failure? And say, “Well, it didn’t work out as you wanted to, the audience wasn’t really that invested in that approach of singing a poem.” There’s a way that that could actually have worked very negatively and inspired a student not to take a chance in this case, but it allowed them the space to try something. So one of the things that I think about with this quote of Ernest Morrell is stop measuring success against failure. What does that mean if we think about grades in regards to equity and inequity? Well, first off, what does grading do? It quantifies our learning. It creates some kind of measure that we’re using as a tool of comparison. It operates, in my opinion, a lot like a credit score. It gives us a numeric value or worth that can grant us access to or limit us from different opportunities. Those opportunities can be admissions into something, whether it be grad school or undergraduate, teaching assistantships, it can offer us opportunities for funding and scholarships. There’s lots of ways that it serves as this gatekeeper process. It’s also something that labels us as learners: the A-student versus the C-student versus the student that we might say is the failing student. And in that way, it can stigmatize pretty extensively. It’s also, in my opinion, a system that both rewards and punishes. And what it’s rewarding and punishing is really dependent on the way that the instructor situates grading in their course, of course. Beyond that, I would argue that, as many other educators working with non-traditional assessment might suggest, that it oversimplifies complex learning processes. It creates a snapshot of a moment that doesn’t give us the full context of what’s happening in that student’s life in that moment. So we don’t have any idea most of the time about all the many struggles that our students are facing, until often maybe it comes about at the last part of the semester, when they’re trying to finally disclose some of the struggles that they’ve had. Aside from that, of course, beyond that comparison, and then like a mechanism of standardization, it also is something that’s applied pretty inconsistently across instructors, which in some ways renders it as a kind of arbitrary measure in the sense that my B isn’t your B. They don’t mean the same thing necessarily. So how are students supposed to make sense of a process where they can’t recognize this standard across this thing that’s supposed to be standardized? Because it invites, I think, when you take all those factors into account together, an opportunity for students to look at this system as a kind of game that they have to navigate. And not a game in a fun way where we’re enhancing our learning, but a game in the sense that, “All I need to do to be successful in this system is to learn what the professor wants. How do I make sure that I meet this deadline or I can amass as many points?” And it becomes this process that for many, I think, is very decoupled from the learning itself. And then we can, of course, see how this snowballs into all kinds of conversations about grade inflation and its relationship to evaluations of instruction and marginalization in regards to the bias that can happen as instructors grade across the different learners in their classes and their abilities and understandings. And all of that is, in the end, we need to start asking that question, I think, more and more: “What do grades mean? What are we actually trying to measure to kind of undo some of the ways in which grades function in an inequitable manner?”

John: And so students spend a lot of their time focusing on learning the rules of the game. But it’s a different game in each and every class that they take, which I think you’re arguing would distract them from actually focusing on learning the material that we’re hoping they get out of the course. Your book is an edited work, where you have a number of contributors. How did you solicit the contributors for this project?

Kristen: I first started by doing an open call in the National Communication Association listserv. At the time, it was called CRITnet, which is Communication Research in Theory Network, it is now called something else. But at the time, I sent out an open call there. I also handed out flyers at the national convention we had in Salt Lake City the year that I was putting together this project. And so I was really excited by the people who responded to that call. I also reached out to a few of the contributors based on work that I knew that they were doing in this area to try to round out the collection. And I can imagine that this could have had multiple volumes, just with the really interesting work people are doing in grading and assessment. So the chapters in the book are diverse in their scope, and even still, they paint a very small picture of a very big conversation. So we had a chapter that focused on grade inflation, just about the rhetoric of grading. Chapters that looked at team teaching and types of collaborative course construction. Assignments that are focused on, I say the “borderlands” or looking even just at activist work in general. And then chapters that explored going into critical Universal Design for Learning, moving into discussions about teacher evaluations, but also just assessment more broadly. And then my own chapters focused on my experiments with non-traditional assessment, the experiments I’ve had in that realm, and the work that I’ve done in addition to setting the stage through the introduction. And I’m so grateful for the collaborators that were part of this project. I mean, I really see it as our book. Of course, my name as the Editor, but we were a team. And we were a team that went about this project in a way that I thought was really beautiful, because I did invite the chapter collaborators to read each other’s chapters and offer feedback as part of the process in addition to my feedback. And we also then worked together at a national convention post the publication of the book to do a short course where we actually taught about our respective chapters. And it was well-attended, and we had some really robust and interesting conversations about how instructors could carry these ideas forward.

Judie: Kristen, in chapter seven you discuss your own experiences with non-traditional assessment. Could you tell us a bit about how your assessment strategies have evolved?

Kristen: I’d love to explore that more. As I said, I feel like my journey with learning to do assessment really started with my work being a student at Goddard College. I was building my own course plans as part of the way that Goddard is set up because it’s working in a tutorial model where, in my case, at the time I was there, you did 15 credits with one faculty member. You built the content for that course and they helped guide you through that process. At the end of that journey, you did a self-evaluation, and they did a narrative evaluation that went into your transcript. So that really, for me, set the foundation for this experience. But then when I went into graduate school, I had the opportunity to work with one of the faculty at the University of South Florida, Mariaelena Bartesaghi, who was, at the time, working with our Interpersonal Communication course. Many TAs oversaw that course. And she had done a grant that she had designed to reimagine that course around process pedagogy and portfolio work, and was really drawing on the work of Peter Elbow and Jan Danielewicz’s work on the unilateral grading contract. And so that was where I really started dipping my toes into finding a way outside of more traditional grading systems when I was a teacher versus a learner. And so I have to say that my own approach started with that approach, the unilateral contract. And for those people that may be unfamiliar with that, it’s a process that really creates the B as a baseline, the B grade. So it’s behavioral in the sense that students are assigned a series of expectations that they must meet in the semester and if they meet those expectations—whether it be number of classes missed, or work turned in, following the instruction, so it’s up to the instructor to determine what those criteria are—then they’re guaranteed that B grade. And anything that goes above a B is up to the discretion of the instructor as far as whether they’re working with a plus/minus, a B+, A-, A. And often in that case, there may be a focus on the quality of the work, there may not be, depending on the instructor and how they implement that plan. One of the things that I found in Elbow and Danielewicz’s system, is that they maintain a strong hold on the course policies. But I was more interested in trying to find a way to go more in the direction of your assured work and thinking about how I could share power with students and have them collaboratively construct course policies. So pretty quickly, and what I kind of call in the text, the guaranteed B approach. One of the ways that I started was always about collaborative course content construction with the students. So consensus process to build ideas about: What does participation mean? How does it function in our class? What role should attendance play in this process? What would an A look like based on the assignments that we’ve done this semester? And so letting them have a voice in that and having a lot of conversations in the context of the class about grading and assessment. So really involving their voice in the process was important to me. I will say that, just kind of broadly, as far as my own approach, I don’t do tests and quizzes, I prioritize other types of assignments. I don’t use percentages or points. Early in my process, I used a lot more markers in my grading. So like a check, check-minus, check-plus, or I would use markers like “meets criteria for a B,” “does not meet criteria for a B,” “exceeds criteria.” Because at the time, I still thought that students really wanted that marker to help keep them on track. And that part has really evolved for me. And I think earlier in my process, I was a lot more attached to attendance and participation models. Even though they might have been student-identified and selected, I put more weight on it than I maybe do at this point. So from the guaranteed B approach, my focus went further into a tiered method that I call kind of a pick-a-plan grading where it was, I think, most similar to what some may use, like a labor-based grading, where you’re actually doing more work for higher grades. And I found that students really responded very positively to that. It still had that kind of guaranteed B if you did this level of work based on the same kind of criteria, but then it allowed more work equals a higher grade. You weren’t guaranteed that more work would equal a higher grade, but it allowed them to make choices upfront saying, “Well this semester, I have a lot going on, so I’m going to choose to do less work,” and that’s okay. And I had quite a lot of students do that. From there, my process evolved to I think what I consider one of my favorite approaches in my experiments over the years was what I called the 100% participation or engagement plan, if you want to use it as a plan, which I think scares a lot of instructors, that idea [LAUGHTER]. Because participation is such a murky realm already for many. What does that even mean? What does that look like in practice? So for me, what that meant in my class is that the students dictated: What does 100% participation look like? And as part of that, I required that students meet with me to discuss their own participation and engagement in the course, we create a consensus around that. And then at the end of the semester, would actually meet to finalize each person’s grade based on what the class determined as their overarching grade categories and the students’ own assessment of their participation. In the text I also talk about the group-focused version of that, which I won’t get into at this space. But the last thing that I talked about as part of my experiment, is that in some semesters, I would just ask the students to pick what kind of assessment they wanted in the class. So I’ve given them some of these ideas: “Do you think you want to do a guaranteed B approach? Do you want to do more of a tiered pick-a-plan approach? Do you want to do an approach where you are doing 100% participation and engagement?” And so it would not be uncommon for me to have some semesters where every class had a different assessment system, based on what the group themselves decided. And key to that is that I always just remained flexible and adaptable to shift it if it wasn’t working, because we had other tools that we could draw from. Another way that the select your own assessment process has worked for me is that you can ask students to decide what they want to be assessed on in the class, this paper versus this—I don’t do tests, but you could do it with tests—as a method. So all of those were really great experiments for me, and “experiment” I almost think it sounds negative in the way I’m saying it because I tried and I had a lot of failures trying these different systems, a lot of struggles, and things that came up. And it occurred to me along the way that maybe some of these practices, while they were kind of masquerading as being more just and equitable, I was maybe falling into some of the exact same traps that I would have been had I been using points and percentages in a more traditional approach. And so that was another huge “aha” moment for me that contributed to the construction of this project. How do I actually embody a system that maybe isn’t falling into some of those traps, even if I think that I’m doing that work? And that’s where I found myself developing my approach to ungrading that I call “awareness pedagogy.”

Judie: That’s interesting how you’re trying to do the right thing, you’re trying to make something better, and then you find out or realize that maybe it’s not better. It’s frustrating. I’ve run through that with offering extra credit, and then I read an article that said that extra credit inherently favors the students who are already doing the best and have the best time and the best preparation. And I thought about that for a long time and realized, “Yeah, I’m doing a disservice to a lot of students by adding to the pressure with more extra credit.” And it wasn’t easy to get to that. I kind of had to see it and then reflect on it for a while, and it was frustrating. So, I don’t do that anymore.

John: Especially when students are asking for extra credit, especially late in the semester. Explaining to them why you don’t do it, though, perhaps could be a useful learning experience for them too.

Kristen: It’s interesting. Because I can’t even remember the last time a student asked me about extra credit in a class. [LAUGHTER] Maybe they just assumed that it just doesn’t exist.

John: Both faculty and students generally find grading to be a very unpleasant experience. And I think many faculty would like to move away from this to some extent. But they may be facing pressures, especially if they’re untenured, and probably especially in the STEM fields, to use traditional grading systems or grading approaches. Are there some strategies that faculty could use that they want to move away from really bad practices to somewhat more equitable practices of assessment?

Kristen: Yeah that’s a great question. And absolutely, I think it’s so important to just acknowledge that not everyone has the same access to actually utilizing a non-traditional assessment in their classroom, whether it be because they’re mandated to assess in a specific way, “Here’s the syllabus, here’s what you’re teaching, go forth and meet your class.” Or if it’s because they maybe are in a marginalized space in the process or they’re in a precarious position. So some of the things that I think that I would invite instructors to consider to lessen the impact of grades and to maybe make the grading process more purposeful is to just, first and foremost, revisit your course policies. And whether you collaboratively construct them with students or not, revisiting our course policies is a really interesting way that we can start to look at the consequences of what we’re setting forward for students to do in our class. So as I said, I used to lean a lot more heavily on the role of late work in my class, like the no late work. And also just having more strict attendance policies because, again, I was thinking about it in terms of this behavioral approach that Elbow and Danielewicz had outlined. And I just stayed with that for a while, and it made sense to me at the time. But after I started seeing that maybe reinforcing these non-academic behaviors was actually not in accord with the learning goals that I have for the class. And I wasn’t really taking into account the whole picture that the student was experiencing, I started to go back and go, “Okay, how can I soften the role that those behavioral policies play in my course?” I think that’s one thing that instructors can do: look at what they’re determining in regard to attendance, late or makeup work, participation in general, and just having conversations with our classes about what it means to participate in the context of this class. What does attendance do or not do in regard to your capacity for success in this course? And also, just more broadly, makeup work and late work is something where we can see a lot of students really suffering in their grades in the end, and it could be that they have any number of things going on that they’re not ready to disclose to us. So that’s one thing. Another thing is aside from course policies in general, and rethinking the non-academic things that we’re actually grading is to consider the possibility of doing minimum grading. As outlined by Thomas Guskey or Douglas Reeves, this idea that eliminating the zero in our gradebook. The zero is a powerful tool that can really keep students from progressing. They may have had something come up that caused them to miss that assignment, maybe were not willing to budge about it, but that zero is going to have a ripple effect that it’s very difficult to recover, depending on the weight or the points of that assignment. So consider minimum grading, a 50 If you’re working with 100 points. What if you give them a 50 versus a zero, that does something that can maybe allow the student some possibility of bouncing back. Because how does it motivate someone if they have a zero and they know they can’t recover? It’s like you’ve lost them. And then what is our learning doing in the class? Are we just going to let them be adrift and not try to find a way that we can move forward? So I would encourage that. I would also say that if we’re thinking about it from a social justice standpoint, that we should stop averaging when possible… again, because if we look at grading as a kind of reward or punishment, averaging can take that moment where we did that zero or we froze them in that time when… Who knows what happened? Maybe they lost a loved one, maybe they just received a very scary diagnosis of their own health, maybe they’ve been working more than 40 hours a week or something that we just don’t know about… they’re experiencing trauma at home or in a relationship. And so averaging, I think, is one of the things that I would encourage instructors to consider reducing their reliance on or even curving. But that, of course, gets us into a whole ‘nother realm of like, “Okay, well, how does that impact now, if you’re going up for rehire, or on the job market, or a tenure and promotion? If you start putting these policies in play, how will your colleagues understand them as they’re assessing you?” Which, in itself, requires a lot of additional labor which I think keeps some instructors from stepping their toes into these different possibilities.

Judie: I read in Jesse Stommel’s blog post, “How to Ungrade,” where they start off their class with maybe the first few weeks the students’ work is not graded, it doesn’t have a point value, it just is for them to learn how to do the assignments and get the feedback. And then they could start from there for some sort of point value, so that they have a chance to grow accustomed to how the course works and what the expectation is before they’re graded.

Kristen: And that’s a beautiful way to think about it too. And that was one of the things that I’m glad you raised it, because I also wanted to mention that, is building in more space for pass/fail opportunities or for ungraded assignments. And even extending that further is maybe building in more space for peer evaluation and self evaluation. Having those conversations with students about what constitutes good peer evaluation or peer feedback and creating guides for that. And even having conversations with your class about what constitutes a good discussion in class, if you’re a discussion-based class, because maybe not a lot of students have actually learned how to engage in productive discussions or dialogues, especially across difficult topics. So creating those opportunities to have space in your calendar to allow those conversations to happen, I think is really, really helpful.

John: You mentioned how students can learn from their mistakes. And as academics, we know that we often learn the most by trying something and failing, and we want to encourage students to do that. But when we use high-stakes exams, that certainly deters students from taking risks and trying new things. What can we do to help relieve some of that pressure to encourage students to be willing to learn from mistakes, because that’s not something they’ve learned from their past educational experiences?

Kristen: Absolutely. And I think that’s why I prioritize revision in my classes, like every course I teach has some element of revision built into the process. So for example, in the courses I’m working with this semester, everything is ungraded up until the final project, which is the primary way in which the course grade is determined. So everything’s a draft until that final project that they turn in. So they have their self-evaluation process as part of that. They have a peer feedback component, and then they get my feedback, they build in that revision. And then we actually do our grade consensus process at the end when they’ve gone through all of that revision and feedback, so that they have that last layer of opportunity to revise it even further towards their grade goal in the semester. And so I think revision is one way that we can do that. And not every class lends itself equally to utilizing a revision project, I realize that, but there are ways that you can do it in a manner that I think students can still gain something from, even if it’s just to show them that not every assignment is one and done, that it’s a process that they can find ways to improve it and to gain new information.

Judie: If we look at the students that are in a classroom, how can faculty leverage the diversity of student backgrounds to create an equitable learning environment?

Kristen: So one of the things that I really wanted to share with you all that I tried this semester for the first time, and I thought the results were pretty great. At the beginning of this semester, I asked my courses to do a syllabi inventory of their classes. And we all know, I know we hear instructors say it all the time, like, “It’s on the syllabus. Return to the syllabus, it’s there.” And it’s this often unread document that creates the roadmap for everything that’s ahead. I know that there’s lots of ways that instructors try and get students to read the syllabus. They create syllabi quizzes, and they do all these little things to get them involved. But I thought, ‘Well, why not ask them to go into their syllabi for all of their courses this semester and to answer some questions?’ So I just used Microsoft forms to build something for them where they went in and they told me about the number of courses they were taking. They did a comparison of the attendance policies across their courses, how participation was defined across their courses. They looked at late work policies, grade grievance policies, policies around accommodation and support, policies that may be focused on communication in the classroom, or specific instructions about how to communicate with your instructor. And not just by email, but how to address them or other things that instructors mention. There were also questions about behavioral focused policies. So what are the things that might cause them to be penalized in a class, whether it be disruptive use of cell phone or technology in the classroom, whatever it is. At the end of that process of looking at the similarities and differences across all their course syllabi, to tell me what their ideal course would be if they were building it based on what they saw in the classes that they’re taking in this semester. And then I use that data in the next class session to say, “Well, how can we build this class to take that information that you’ve gained and to create policies that would be compassionate, but also hold you accountable for your learning choices so that you’re getting the most out of this class that you can?” And it just was such a fun conversation. And I got a lot of feedback from students that was unsolicited in the sense that they said, “Well, I just had never looked at my syllabi in this way before. And I actually feel a lot more prepared for the semester now that I actually compared.” And it gave me a sense of just, like, how much stress students face trying to navigate the different instructor expectations. I was, I don’t want to say shocked, because we’ve been doing this for a long time and you have a sense because you talk to your colleagues about what they’re doing, but just the level of work, expectations that were there, and the huge spectrum from very flexible to very inflexible, and how it would be a full-time job for students to just navigate those expectations. So it makes sense to me even more now that we’re maybe putting our emphasis in areas that we could rethink, as educators, to help students get the most out of their learning, and less about having to make sense of what we want in a class.

Judie: I can relate to that, because I teach history online, and I keep weekly schedules. But if students need more time, they just have more time, they have until a date at the end of the semester when everything is due. And I try to re-emphasize that you take the time you need, it’s fine, there’s no “late,” there’s no penalty, just relax. And when you can do it, you do it. But then I send them reminders that this phase is ending, this next one is starting. So it’s a good idea to try to stay on track. And often I’ll get emails from students saying, “I’m so sorry, this is late. I understand there might be penalties.” And I think, “Why do they understand there’ll be penalties?” And all sorts of apologies. But then I kind of took a step back and thought, “Oh my gosh, if they’re juggling five and six classes and all these different policies, of course they’re confused.” And I just try to write back and reassure them that I understand that people have different situations, and you have to take the time you need without penalty. And please, don’t let my dates add to your stress. But it’s got to be really difficult for students to try to keep track of everybody’s policy on top of all the reading and work that they have to do in all their courses.

Kristen: Absolutely, yeah. It was a real eye-opener for me to just see the data in front of me. And to contextualize that with the broader conversations we had about just their general relationship with grades and grading and their own perceptions of whether grades accurately reflected their learning in their classes. So it invited us into a space that I thought was vulnerable, but also really powerful for imagining a way to do it differently. And of course, we have this backdrop that we’re facing with the pandemic and how campuses are navigating the return from remote learning to in-person instruction and the stresses that come with that as students maybe are now navigating not only different policies, but different platforms. So that was another question that I asked is, “How many classes are you doing remotely versus in person? And how is that impacting knowing what you’re doing and when and where and how in your process?” Again, a lot of stress. As a new Chair, I can say that I have had so many conversations with students this semester, just in tears, trying to make sense of maybe unclear expectations that are being set forward in their courses, or just lack of communication that’s happening. And I just get this sense that so many of them feel adrift. And I know that, at least among my colleagues on campus, our motivation has been challenged because you go out and into your classes and you maybe see that people aren’t as engaged or connected as maybe previously pre-pandemic. And it’s like you feel like you’re tap dancing really vigorously to get everyone to be part of a process, and it’s this delicate dance we’re all doing to make this matter.

John: You started this project before the pandemic but it was completed during the pandemic. How did the pandemic influence the final work on the book? And do you think the experience that faculty had in more directly observing some of the challenges our students faced might make them more open to considering non-traditional grading practices?

Kristen: I definitely think it has made faculty more open. I’m part of a lot of social media pedagogy-focused groups where there’s been pretty strong debates about what we’re doing in this moment as we teach and learn in a pandemic. And some people feel pretty strongly about maintaining this perception of rigor and these strong standards as a way to keep everyone on track and hold on to that perceived norm that we had pre-pandemic. And then there’s others that have, I think, done so much emotional labor, bending over backwards to be as compassionate as possible to recognize just the weight that everyone’s carrying in regard to just the heaviness of this pandemic, and the impact it’s had on us personally, professionally, and just socially. And so, especially at the earlier stages as this book was coming out, I wanted to go back in before it actually went to print to talk about the ways in which institutions had transitioned to different grading models in 2020, to try and attend to the impact the pandemic was having. So we learned that, institutionally, while it seems like you can’t decouple traditional grading systems from academia in general. We did. We went into so many institutions, created pass/fail options, credit/no-credit options, a variety of different system-based changes where students could not have their GPA directly impacted by the pandemic. And then, of course, we saw that happen, and then we went right back to the previous methods and models pretty quickly after that semester, returning to this norm. So I say “norm” kind of in air quotes, but it reminded me that we can do it, we can make some transformative changes in our learning if we want to collectively embrace that. But it’s also something that I think people still have a lot of discomfort about. I think many instructors, at least I hope, want their students to succeed, and they want to be compassionate and to help them succeed. But we don’t always know the best way to do that because we are managing, ourselves, a lot of expectations just in our own responsibilities and roles. And we’re also tired, and many people are stressed, and just definitely surviving, not thriving, in this moment. So I know that people are also getting fatigued… compassion fatigue happening. They’re becoming a little bit less trusting of the many emails that students are sending, asking for exemptions and extensions and extra credit. And so I think we’re in this moment where we’re all invited to say, “Where do we want to go from here? What kind of learning model will best meet the needs of our future generations because so much is impacting it?” We have this huge political opposition that’s permeating our social world and conversations in the public sphere. We have this fear and anxiety about climate change. We have just so many things going on that this is a beautiful moment for us to imagine a new way forward that could best meet everyone’s needs, I hope, to thrive more in our learning environments.

John: Behavioral economists have found a lot of evidence of status quo bias, that people tend to do the same things in the same way, unless there’s some sort of disruption. And I think this pandemic, and all the other things you mentioned, have led to a disruption which makes possible transformative change in ways that would be much less likely to occur at the same rate in other time periods. I’m hoping, at least.

Kristen: Exactly, and I know I think about it a lot in just regards to changing our own communication patterns in our relationships. I mean, one of the ways that we go about doing that is to do what comes unnaturally, to do the opposite… cultivate the opposite, jostle ourselves out of our norm so that we can imagine another possibility. So I’m hopeful. I think that we’re seeing just these conversations just taking hold in a lot of ways. I mean, I lean a lot on the Facebook group, Teachers Throwing Out Grades. It’s a big group of people, about 12,000 people at this point, that are having these conversations just in that one space. I mean, I know they’re happening in all kinds of other spaces. But that one is one that I like to follow very closely. And even, if we’re using Facebook as an example, the Pandemic Pedagogy group is where you see a lot of people having debates about these issues that we’re facing as teachers and learners.

John: Yeah, those groups have been really helpful in the last year and a half or so, as well as the Twitter conversations.

Kristen: Oh, absolutely. Yes, the academic Twitter and other spaces.

John: And we’re recording this at the end of the semester. And you mentioned all of the emails and requests from students. And one thing I’ve tried to convey to my students is, unless there’s really extraordinary circumstances, I’m not going to make special exceptions only for the students that approach me, that I’d rather build it into the course structure itself so that those opportunities are available for everyone. Because, otherwise, the students who are most likely to request extra credit and so forth are the students who generally come from continuing generation families. And there’s a lot of students who don’t realize that they have that opportunity to request things. So in general I try, in my courses, increasingly in the last few years, to build in more opportunities for revision, for submitting things late, and so forth. But those are open to everyone on an equal footing. And I say if they’re having some major crisis, then I’m happy to talk about it. But in general, I think we have to be careful not to only make exemptions for those students who come forward. It’s much better, I think, to build those opportunities for everyone, including those who might be afraid to ask for those special cases.

Kristen: And I agree, and I definitely try to structure my courses in that way. But I’m thinking now from kind of the perspective as a Chair or just as a Faculty Advisor… What do we do with the students that, in those classes where they have those opportunities, they’re still succeeding because those opportunities exist, but they also, maybe in three of their five classes, are dealing with these very rigid policies that maybe instructors are not understanding that they just got a major medical diagnosis, or they’re on the verge of needing to take a medical withdrawal because they’re having a mental health crisis? And then how do we help those students still succeed? And those are questions that I don’t have an answer for. I keep asking myself that because I know that, when I think about grading broadly, often grading is a disruptive tool that impacts the relationship between the teacher and the student. And so maybe students don’t always feel comfortable coming and talking about it until the end where they’re saying, “I didn’t get the grade I was seeking,” perhaps, or, “Is there anything I can do to recover the grade?” But then where can we plug in in other spaces, I think, instead of being in that instructor role, but in advisor roles or in other, like, role-model positions with students where we can help them? Those are questions I keep asking.

John: Certainly there’s a difference in instructor flexibility, which is a major problem that students face. And as we talked about before, they have very different requirements in each of their classes. And just yesterday, I had five requests for extra credit. And each time I referred them to the opportunities that were already built in. And I said, “I’m not going to ask you to do extra work when you haven’t done some of the required work that you still can do. Before you ask to do something more, maybe you should look at the things that are available for you that you’ve been asked to do since the start of the semester and start there.” And that doesn’t always get the most positive response.

Kristen: And that’s where I think self-assessment can be a really great tool that, if instructors actually build in, whether it be on an assignment basis, or just the broader course, a self-assessment. I know that I’ve worked with asking students to kind of keep a log of their process in the class, the work that they’re doing in-person or out of class. If you’re remote, of course, those distinctions aren’t important. But it’s been helpful, I think, for us to have those honest conversations at the end of the semester. It’s like when we’re talking and you’re saying, “Well, I think I deserve an A in this class.” But then I’m saying, “Well, but you weren’t there for more than 50% of the semester. And these are the assignments that were not turned in. Please help me understand your perspective so that I can say, ‘How is that fair for those students that maybe have been there and participating in a way that you weren’t. Help me understand how you’re understanding this.’” [LAUGHTER] And then I would say 9 out of 10 times that student comes back and says, “Wow, that probably isn’t fair to the other people.” So anytime that I think we can pull back the curtain and just have these process conversations, I just continue to be so inspired by what can come out of them. Of course, you’re always going to have the student that’s like, “I deserved an A. I know I didn’t turn anything in and I wasn’t there.” Because I think they’ve learned that in the game, or the rules of the game, that if they just keep self-advocating for the A, that maybe we’ll somehow meet them in this, like, maybe you’re failing them and they get the A, maybe, and you’ll land at the C or something. And they see it as this negotiation practice. But more often than not, I would say I have found that, at least in my classes, that students actually are harder on themselves in their self-assessments than I would have even been.

John: And at the other extreme though, might stereotype threat play a role in some of the self-assessments as well, for students who are in marginalized groups?

Kristen: Absolutely. One, you could say the idea that many people have an inflated sense of their effort, or their knowledge of a topic, right? So we have that. And then they just hold to it. And then we have the ways in which we’ve embodied these negative stereotypes and stories about who we are as people and learners in different identity groups. And of course, that’s going to impact. That’s why the thinking about difference and how difference punctuates every part of our process is so vital, right? It’s a difference, because we all come to the table with different capacities. I think this is why, more than anything, I started asking in my self-assessments, I mean, the most important thing for me to know is, “What was most meaningful to you in this class? How are you going to build that into your life in some capacity and take it forward?” So it doesn’t mean that I’m not chasing concept understanding, it just means it’s more for me about what matters to them, and so it just changes the assessment conversation. If they can’t articulate what’s meaningful to them, that tells a pretty specific story. That’s quite different than the student that says, “I’m not such a good writer, or I had all these struggles that impacted my capacity to turn work in in this way or that way or to participate as much as I would have liked to because I have a lot of anxiety and I don’t feel comfortable speaking in front of the class. But here’s what mattered to me, here’s what I’m going to say. This has changed who I am as a person, because it changed my thinking.” That’s what I want to assess. That’s what I want to know, as a teacher.

John: And that type of metacognitive reflection, we know, helps increase learning, and there’s a lot of research to support that. So, by itself, that’s a really good practice to encourage. So we always end with the question, “What’s next?”

Kristen: So what’s next for me is I’m hoping to continue to have these conversations and continue to experiment in my classes with ways forward that I can refine what it means to do assessment and grading from a social-justice perspective. How we can best harness our communication resources—whether it be our theories, or our methods, our conversations in the classroom—to create a system that’s more just in the realm of teaching and learning. So what I’m really working on now is expanding this thing I’m calling “awareness pedagogy,” which is something I wrote about in the book in chapter eight. This process that’s built out of ungrading. Ungrading is an umbrella term that I think has been pretty widely adopted by many people to talk about a type of grading process that decouples grading from feedback. And it really focuses very heavily on learner self-assessment. In many cases, for people doing ungrading, that means that the learner themselves assigns their grade through a self-assessment process with the instructor’s support and sometimes integrating peer feedback. So, in my case, awareness pedagogy, I’m using five broad categories that I work with to help students build awareness in my classes across the kind of courses that I teach. And I teach classes in Communication, I’m doing classes on listening and interpersonal communication, intercultural communication, classes focused on conflict. And so it’s really well-suited for that discipline, which I know is different than other people that may be listening. So I’m working on that. My next project is to expand what I introduced in chapter eight into a book-length project to really get into the nuts and bolts of awareness pedagogy as its own kind of approach to social justice assessment in the classroom and what that looks like. Especially in the realm of thinking about diversity, equity, inclusion, and access, but also integrating a lot of contemplative pedagogy as part of it because that’s an area that I’m extremely attracted to in my own work.

John: Well, thank you. And it’s been great talking to you, and thank you for all of your work on behalf of students.

Judie: Thank you.

Kristen: Thank you so much. It’s been wonderful to spend this time with you all.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Anna Croyle.

[MUSIC]

210. A Pedagogy of Kindness

The informal culture of some academic departments can facilitate an atmosphere of mutual mistrust between faculty and students. In this episode, Cate Denial joins us to discuss how a culture of suspicion can be replaced by a pedagogy of kindness. Cate is the Bright Distinguished Professor of the History Department and the Director of the Bright Institute at Knox College in Galesburg, Illinois. Cate is the 2018 to 2021 Distinguished Lecturer for the Organization of American Historians and the recipient of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award. She is the author of A Pedagogy of Kindness, which will be released as part of the West Virginia University Press’ superb series of books on teaching and learning.

Shownotes

Transcript

John: The informal culture of some academic departments can facilitate an atmosphere of mutual mistrust between faculty and students. In this episode, we discuss how a culture of suspicion can be replaced by a pedagogy of kindness.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Cate Denial. Cate is the Bright Distinguished Professor of the History Department and the Director of the Bright Institute at Knox College in Galesburg, Illinois. Cate is the 2018 to 2021 Distinguished Lecturer for the Organization of American Historians and the recipient of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award. She is the author of A Pedagogy of Kindness, which will be released as part of the West Virginia University Press’ superb series of books on teaching and learning. Welcome, Cate.

Cate: Thanks for having me.

John: We’re really pleased to have you here. Many of our guests have referenced you on past podcasts. And you’ve long been on our list of people to invite so we finally got around to that. I’m sorry it’s taken this long.

Cate: Oh, I’m glad to be here now.

John: Our teas today are… Cate, are you drinking tea?

Cate: I am drinking tea. I am drinking Yorkshire Gold black tea with just a hint of milk in it.

Rebecca: The true British way.

Cate: Exactly, it’s the way of my people. [LAUGHTER]

John: We long have had some of that stocked in our office for our British faculty members because that tends to be pretty much universally their preference.

Rebecca: I have that East Frisian, that’s my new favorite.

Cate: Ooh.

Rebecca: It’s a black blend, of what I don’t know.

John: And I have a pineapple ginger green tea.

John: We invited you here to discuss “A Pedagogy of Kindness.” You’re working on a book version of this now, which grew out of a document you posted on Hybrid Pedagogy in August 2019, and it’s been well referenced by many people. It’s been a useful resource, especially during this pandemic. In this blog post, you talk about your evolution as an instructor. Could you give us an overview of how your teaching approach changed after you attended that Digital Pedagogy Lab Institute at the University of Mary Washington that helped prompt some of these changes?

Cate: Yeah. The Digital Pedagogy Lab Institute in 2017 was kind of the “aha” moment for me. And events had been building up to that for a while. So when I was a graduate student, I was not taught how to teach very well. And I was sort of taught to think of students as my antagonists, to anticipate that they would try and get away with all kinds of things, they would plagiarize, they would cheat, they wouldn’t show up to class, or do the reading. And that my teaching career has been the process of unlearning all of those things. I have been helped along the way by colleagues in K-12 education when I worked with the Teaching American History grant in Iowa for 10 years, by some of my colleagues from Knox College, particularly Gabrielel Raley-Karlin, who is my friend and associate in sociology. And then I also was a participant in some intergroup dialogue workshops at the University of Michigan. And all of those things kind of came together to sort of make fertile ground for the stuff at DPL to sort of land. The Digital Pedagogy Lab is a profoundly kind place, everybody is so well taken care of, there are pronoun buttons, there’s great food, all of your creature comforts are taken care of. And this track that I was in, which was the introductory track, was very focused on how to really care about our students and to interrogate the way that we taught to ask if we were sort of thinking about their needs fully. While I was there that weekend, I came to the conclusion, I had this moment of going, “Why not just be kind?” and that really set me off on this new trajectory.

John: What were some of the practices that you had been using that you moved away from as a result of this Institute?

Cate: I took a long, hard look at my syllabus, and really noticed that the language in which I was speaking to students was very much from a place of authority, sort of on a pedestal, instead of thinking of them as my collaborators. So, I changed the way that I talked about all the policies on that syllabus. I changed the way that I talked about the honor code from being very sort of finger-waggy and sort of insinuating that everyone was going to screw up at some point, to a statement that said, “Hey, I take responsibility for teaching you how to do these things. And I believe that everyone in this class is fundamentally honest,” which is completely 180 from the language I was using before. I stopped taking attendance, I stopped having hard deadlines for assignments of any kind, I became infinitely more flexible with my students. I changed the “I” statements in my syllabus to “we” statements and really emphasized that I thought of students as my collaborators. Everything changed. Everything changed because I looked at it from a completely different vantage point after that moment.

Rebecca: I think one of the things that comes up when we say “kindness” is that people confuse that with “being nice,” or just being a pushover, having no standards.

Cate: [LAUGHTER] Yes.

Rebecca: …a laundry list of things that are associated. Can you talk about what you mean by “being kind”?

Cate: Kindness is definitely not niceness. I like to say that niceness is okay with lying, and kindness is not, it is unkind to lie to someone. And kindness often means telling very hard truths. But kindness is about three things in teaching, I believe. The first is justice, the second is believing people, and the third is believing in people. So justice means knowing who is in a classroom and who isn’t at any given time. Being super attentive to our positionality, and thinking about our social identities, and those of our students. Thinking about student needs in all their complexity. So having a basic needs statement in my syllabus, making sure I have fidget toys for students, I bring a huge bag of snacks to class, those kinds of things. So really thinking carefully and honestly about where I’m standing and where they’re standing. Believing people means that when people tell me that their printer died, their dog ate their homework, they had the flu, that I believe that on every score. I always feel that it is better to risk the idea that someone might pull one over on me, than to inflict more hurt on a student who’s already in crisis. So I always err on the side of belief. And then believing in people means believing that students can be our collaborators. So, changing the way that I grade so that my students and I do that together, changing the way that I think about our conversations as a class when we’re doing class discussion, and structuring those to make sure that everybody feels heard. Making sure that students get a say in what we read and what direction the course goes. And all of those things, I think, are integral to showing compassion and making the classroom a compassionate space.

Rebecca: I’d like to pick up on one of the ideas that you just presented, which was this idea of grading with students, not something we often hear. Can you talk a little bit about what that looks like?

Cate: Yeah. So, I’m a big proponent of ungrading, and ungrading is basically a big umbrella term for any action that gets us away from having numbers and letters on assignments, at any point. So there’s a big spectrum, you can do very small things that contribute to an ungrading atmosphere, and you can get rid of grades altogether if your college supports you in that. So, at my institution, what I’ve done is my students and I put together a list of grading standards, things that we think constitute each of the grades on the grade spectrum. And then when they turn in their first paper, they also turn in a self-evaluation of their work. And some of the questions are very mechanical: Did you turn it in on time? Did you ask to turn it in late? Did you do what the assignment prompt said? And some are much more open-ended: In what ways was this assignment an act of exploring new intellectual territory? I always end the self-evaluation with: “Is there anything else I should know?” …which is a great space for students to be able to tell me all the myriad things that are going on as they’re trying to focus on this assignment. And then the students and I either sit down together or Zoom together to have a conversation about what they think their grades should be. And sometimes we reference those standards that we talked about already in class, and then what I think perhaps their grades should be, and we discuss it. We talk about what are the two big things they could do that would make their assignment even better. And we focus on what you can do next. So we come to an agreement about what a grade should be. And my role in that is, really, to make sure that people don’t undersell themselves, and to make sure that people are accurately summing up the work that they did, rather than, some students have said to me before, “I don’t want to seem conceited by saying I get an A.” So there’s all kinds of little hiccups that I have to take into account.

John: I think a lot of faculty resistance to ungrading deals with those two extremes with students who may undervalue their work and students who overvalue it. Do these discussions with students help correct their perceptions and help give them a better understanding of what they’ve actually learned?

Cate: I think so. And I think that having the conversation about grading standards before we even get to awarding a grade is a really integral part of the process. So we co-create those standards, and they get to say if they want to edit a line, take something out, put something in. So we’ve already had a really great conversation about what grading is, and why I approach grading this way, before we ever get to the point where we’re going to grade an assignment together. In the four years that I’ve been doing this…a little over four years now… I have never had someone overestimate their abilities. But I have had many students who have underestimated their abilities for a variety of reasons. And so, it’s great to be able to say, like, “I think you’re underestimating yourself, let’s bump that up.” And to explain why, also, so that they have a better sense going forward of what they’ve achieved, and what they can continue to achieve.

Rebecca: One of the things that you also highlighted, Cate, is the idea of flexibility. And I think the phrase you used was “infinite flexibility.” [LAUGHTER]

Cate: Yeah. [LAUGHTER]

Rebecca: I can imagine many faculty really running away from the idea. [LAUGHTER] Can you talk a little bit about why you shifted to flexibility and what you mean by flexibility?

Cate: Yeah. When I say “infinite flexibility,” I don’t mean you have no boundaries and you do whatever. What I mean is more, there are so many ways in which I’ve been called upon to be flexible, that I couldn’t possibly enumerate them. There are different kinds of ways every time. So, what do I mean by flexibility? I mean, I plan my courses so that I have the time to be able to have softer deadlines, for example. So I set aside some time for these grading conversations in the week that a paper is due, but also in the week after because I know there will be students who need extensions. And so I make sure that I have plenty of time to be able to have those conversations with them, no matter when they’re going to turn in that paper. It means fostering an atmosphere where students feel okay saying, “Something’s come up, I really need that extra couple of days.” And I don’t force disclosure, so no student needs to tell me exactly what is going on in their life in order to get that extension. It’s really just a question of saying, “Hey, I need this thing,” and then it’s theirs. Flexibility in readings, being able to change things on the fly as things are revealed to be too easy, too hard, finding exactly the right mix for a particular community of students. Flexibility about time off. When do we all just need a break and a mental health day? Those kinds of things. Flexibility in terms of the kinds of assignments that I make, and the things that I asked students to do for a grade. That’s really important too, I think.

John: You mentioned giving students more ownership of the course and that flexibility certainly would be part of that. But you also, I believe, talked about using UDL principles in your class. Could you talk about some of the ways in which you’ve implemented UDL strategies?

Cate: Yeah. So the design of my syllabus is something that I thought a lot about on that score, in terms of making sure there’s always alt text where I have images, making sure that there are images that help guide people to certain pieces of information so that it’s not a wall of text that faces a student at any one time. Making sure that there’s lots of resources in the syllabus for students who might need extra help, whether that’s tutoring or talking to me, or connecting with our office for disabilities. In the classroom it has meant things like bringing in a large basket of fidget toys and encouraging people to use them. Making sure that, wherever possible, I have both a text version of something, and an audio version of something if it’s available. Making sure that if there is an audio version of something, there’s a transcript. Making sure if I’m uploading videos, that those are transcribed also and have subtitles. So all of these things adjust with me, trying to keep in mind: How can I reach the maximum number of students as possible at all times knowing that many of my students may have things like learning disabilities, but they’ve never been diagnosed? Or, they can’t afford to have them diagnosed. So planning things so that I try and catch as many things as possible that I can anticipate, and then being flexible with other requests as students make them of me.

John: Do you give students multiple ways of demonstrating their learning?

Cate: Yes, I do. So, one of my favorite assignments is the unessay. So that is an assignment where I ask students to show me what they have learned in any way that does not involve a major paper. I used to give them the option of a paper or something else. But I found that people often chose the paper because they thought it was the safer route. And what I was interested in was getting them sort of outside of that thinking and trying something experimental. So I’ve had students make food and diaries, and do embroidery, and make quilts and dioramas and maps and street plans, and just an amazing variety of ways to show me what they’ve learned in a given term. That also means that they can tailor that to what they are best at, right? So there are other assignments in the term that are written papers. So this assignment, if you’re someone who doesn’t write papers well or really struggles to write them well, this is a moment for you to show me that you can rap, or you can sing, or you can play guitar, or you can make something. And for the students who really find papers easy, this is a moment to refine another skill, to get really good at making a presentation, for example, or to think about how to visually communicate their knowledge. So I think it has something for everybody.

John: And for you, I imagine, it’s much more fun to listen to these different forms of assignments.

Cate: It’s super fun. And their creativity just astounds me every time that I do the unessay and I do the unessay in almost all my classes. I would not have thought, for example, to make a star quilt in a course about native history. But one of my students decided to research the kind of sewing that students were asked to do at some of the boarding schools, and found the long history of star quilts in native culture, and then decided to make a very simple one for themselves. That was a tremendous project where they learned so much about native history in the 20th century, and I would never have predicted that in a million years. Their vision of what they can do is so much bigger than what I can imagine on my own. And that’s one of the real delights of the unessay, is getting to find out all the other things that they’re good at, and all the ways they can draw connections to places around campus, other things they’re doing, other disciplines that they’re really interested in.

Rebecca: Cate, when you have students complete an assignment like an unessay, is there some sort of companion to go with that to explain the learning that occurred while they were doing that activity?

Cate: Yes. So there are a couple of other pieces that go with it. The first is that when the students make a proposal to me for what their unessay will be, they also have to turn in our grading standards modified for that project. So that’s another place where the grading standards come in really useful. That means that when I’m going to grade everything, I have an individualized grading sheet for every single project and can sort of just go through them one by one. Students also turn in a reflective paper where they reflect on what they learned by making or doing their project. And those are some of the best pieces of writing that I get to read. They’re much more informal than a paper would be. But they are these wonderful spaces where students are incredibly honest about where they struggled, and how they overcame those struggles, and what the projects have meant to them, which is really exciting.

Rebecca: For those reflective assignments, do you have specific prompts that you encourage students to respond to, or is it more open than that?

Cate: It’s much more open. I just say, “You know, I want you to reflect on what you learned during this process.” And they can take that in any direction that they want.

Rebecca: One of the other things that you brought up, in terms of flexibility, were less rigid deadlines. But a lot of faculty are often very concerned about workload or other things that could occur if the deadlines were relaxed. Can you talk a little bit about how you manage your time with this flexibility? You’ve talked a little bit about the conferencing and making sure you have conference time, but when you’re getting many things in over the course of the semester, how do you manage that?

Cate: I have reduced the number of assignments that I ask students to complete. I used to have many, many more. And I realized that some of that work was busy work, and that I would rather have fewer assignments that took longer, and where students were more engaged than lots of little bitty assignments throughout the term. Some of it is planning, some of it is planning to give myself a different kind of time to grade these things. So, in the grading conversations, like I said earlier, being able to have sort of time spread over two weeks, instead of just one, to get everything graded. It’s also about talking to students about exactly how much time they need to get the assignment done. So you raised the question of workload like, “Aren’t we going to add to students workload and their stress if they’re just putting these things off?” But what I found is that I can’t predict when their workload is highest. And sometimes my assignments really make for a crunch for them, because everybody’s expecting everything at the same time, such as around midterms. So saying to a student who asked for an extension, “How much time do you need?” Then perhaps a conversation where we can say, “I just need a day,” or “I need two.” And it never becomes a situation where I’m like, “Turn it in whenever you want.” [LAUGHTER] It’s much more about, like, “Let’s realistically think about what extra time would be useful to you, without it becoming an open-ended thing that can drag on forever, and really become a problem.”

Rebecca: That’s a really important point because having infinite deadlines is not helpful for anyone. It’s not helpful for us as instructors, and it’s not helpful for students.

Cate: Absolutely.

Rebecca: We all get motivated by deadlines, even if they are a little flexible. And as professionals, we know that our deadlines are a bit flexible, often.

Cate: Exactly, yeah.

John: So, do you think that pandemic has made people more open to consider a pedagogy of kindness as they’ve observed some of the struggles more directly of our students?

Cate: I think that has been the case, yes. I think there is tremendous momentum towards pedagogies of care. I think that we’ve also experienced the pandemic for ourselves. And we have been overworked and stressed out and worried about our families and friends and communities. And we have needed kindness, we have needed the breathing room that this can provide. So, I think that it is both seeing the real challenges our students face often because for the first time we were inside their homes, and seeing some of the material circumstances that they were living in, hearing from them about the challenges they were facing mentally and physically, but also reflecting on our own experiences and knowing what would help us. And we didn’t always get that help ourselves. And so being able to provide it for others, I think, has been a really good thing.

John: What is the anticipated publication date for your book?

Cate: I don’t know. And that is because I needed a little kindness myself this summer, and for my deadline to be a little bit flexible. My original due date was September 1st for delivering the manuscript but I had some major health challenges this summer. And so I wrote and asked if I could have some more time, and I was very glad to be working with an editorial team that was great, and that gave me that extra time. So, the book manuscript will be delivered this Fall, but I’m not sure where that will put things in terms of a publication schedule.

John: In the meanwhile, your “Pedagogy of Kindness” blog post is available to anyone who would like to read it, and it’s a very useful resource. And you’re joining a great collection of books there, we’ve had many of the authors on and we’ve referred to these books very often. And we’ve used many of them for our reading groups, and we share many of them with our faculty.

Cate: Yeah, I was once given the advice by one of my advisors that, when you’re thinking of publishing somewhere, look on your bookshelf and see where all the rest of the books come from. And when I looked at my bookshelf on pedagogy, everything was coming out of West Virginia University Press, and so I knew that that was exactly where I needed to pitch my book.

Rebecca: I know you have many people waiting for it, and we’re all excited to read it.

Cate: I’m very excited to finish it, so…[LAUGHTER]

Rebecca: I’m sure. The best part about having things on a to-do list is crossing them off. [LAUGHTER]

Cate: Exactly, yes.

John: You’ve been running the Bright Institute for a while, could you tell us a little bit about this?

Cate: So the Bright Institute was something I came up with five or so years ago. We got a very generous donation at Knox College, from the family of Edwin and Elizabeth Bright. And it was to facilitate the teaching of history before 1848… American history before 1848. So my idea was to bring together other liberal arts professors from across the country who also focused on that time period, so that we could try and help people with some of the challenges that liberal arts professors face. So we generally have less time to devote to our scholarship and to keeping up with readings in the field. We tend to not get grants or fellowships at quite the same rate as our colleagues at big research institutions. And we are people who have a lot of responsibility for teaching. So the format of the Bright Institute is that every summer there is a two-week seminar. The first seven days of that seminar are about reading scholarship in some particular field within early American history. We’ve had some just incredible conversations in those parts of the seminar. And then the last three days of the seminar are devoted to pedagogy. So taking the content knowledge that we now have, and thinking about, “How do we apply that to the classroom situation?” And then to help with research, we give everybody who’s a part of the Institute $3,000 every year to fund their research or to take them to conferences, there’s lots of ways that people have used that money to support them in this scholarship.

Rebecca: That looks like something to look forward to every summer.

Cate: Yes, one of the highlights of my career [LAUGHTER] is to be able to support so many people in doing such incredible work. And it’s such a delight to bring everybody to Galesburg every summer and have 14 other people who all do the kind of history I do. We tend to be kind of isolated on our campuses, we’re very often the only person who does early American history. And so to have this wonderful team of people with whom you can talk about scholarship and teaching is just so filling.

John: I wish we had more of that in all disciplines.

Cate: Me too. And I wish that I could replicate this… like I personally had the funding to replicate this for say, community college people, for precarious academics. It seems to be working very well, and I would love to see that model replicated in other ways.

Rebecca: Well, thanks so much for all this information and things to think about as we’re moving into next semesters, next classes, next academic years. We always wrap up by asking, “What’s next?”

Cate: What’s next, most immediately, is finishing the manuscript and getting that off to my press. And then after that, my college just won an NEH grant. So next summer, I will be leading a team of students in researching the dispossession of native nations from what is currently called West Central Illinois, building out on a website that some students and I have already built, and going to visit the communities that were dispossessed, to build relationships between the college and those communities. So that’s a really exciting thing to have on the horizon for next summer.

Rebecca: That sounds really exciting.

Cate: Yeah, it is. And we just found out about it, so it’s brand new information. [LAUGHTER]

John: That’s wonderful news, congratulations.

Cate: Thank you.

John: We’ve really enjoyed talking to you. We’ve been looking forward to doing that for a while and thank you for joining us.

Cate: Thanks so much for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Anna Croyle.

[MUSIC]

187. Talking Tech

Student use of mobile technology can enrich student learning experiences, but can also interfere with the focused attention that is essential for learning. In this episode, Michelle Miller examine how we can talk to students about technology in ways that will help them become more efficient in their learning and professional lives.

Michelle is a Professor of Psychological Sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology research. Her research interests include memory, attention and student success in the early college career.

Michelle is the author of Minds Online: Teaching Effectively with Technology, and has written about evidence-based pedagogy in scholarly as well as general interest publications. She’s currently working on her newest book, Remembering and Forgetting in the Age of Technology: What the Science of Memory Tells us about Teaching and Learning in a Wired World, scheduled as part of the West Virginia University series on teaching and learning.

Shownotes

Transcript

John: Student use of mobile technology can enrich student learning experiences, but can also interfere with the focused attention that is essential for learning. In this episode, we examine how we can talk to students about technology in ways that will help them become more efficient in their learning and professional lives.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Michelle Miller. Michelle is a Professor of Psychological Sciences and a President’s Distinguished Teaching Fellow at Northern Arizona University. Dr. Miller’s academic background is in cognitive psychology research. Her research interests include memory, attention, and student success in the early college career. Michelle is the author of Minds Online: Teaching Effectively with Technology, and has written about evidence-based pedagogy in scholarly as well as general interest publications. She’s currently working on her newest book, Remembering and Forgetting in the Age of Technology: What the Science of Memory Tells us about Teaching and Learning in a Wired World, scheduled as part of the West Virginia University series on teaching and learning. Welcome back, Michelle.

Michelle: Hi, it’s great to be here.

Rebecca: So good to have you back. Today’s teas are…. Michelle, are you drinking any tea?

Michelle: Well, I’m still on coffee. We have a three hour time difference this time of the year. And so I figure I’m entitled.

Rebecca: How about you, John?

John: I am drinking ginger peach green tea.

Rebecca: And I have golden monkey today.

John: …for a change.

Rebecca: It’s expensive. I only drink it on special occasions. I was like, we’re gonna get to talk to Michelle, today. I’m gonna make fancy tea.

Michelle: Well, coffee is the fanciest tea of all.

John: We’ve invited you here to talk a little bit about how to talk to students about technology and why perhaps you might consider talking to students about technology. You teach a course on mind, brain and technology, and you’ve also created the Attention Matters projects that we’ve discussed on an earlier podcast. Could you tell us a little bit about the mind, brain and technology class that you teach?

Michelle: Right. So this has been such an incredible privilege I’ve had, on and off. for several years. Now, back a long time ago, when I first applied for and was competing for the President’s Distinguished Teaching Fellowship honor and award here at Northern Arizona University, one of the things that we got to do as part of our application packet was to envision a dream course. And this was, gosh, around a decade ago that I did this. So the landscape of the research and technology itself was very different. But this is the course that I came up with to say if I could teach one thing, brand new, build it from the ground up, this is what I would do: something that would connect psychology, especially empirical research-oriented psychology, the role of emerging technologies in our lives and the incursions they’ve made into all of our lives, and blend that with some real practical advice and things that would be engaging to college students today at a variety of levels. And so it went in my packet. I was so fortunate to win the award and be chosen for it. And then I came knocking on the doors, and I said, but remember, there was this dream course, I actually was very literal minded. So I said, “Well, I get to teach this now, right?” And my department said “Well, oh, okay, yes, we can work that out.” And it originally was taught as a senior capstone, and it’s been taught in that form, again. Another time it had an incarnation as a freshman seminar, a first-year seminar, and right now I’m teaching it as a fairly large general elective upper division elective, primarily serving our psychology majors and our minors. And so this is a course that I’ve been able to dip in and out of throughout the years. And I actually quote one of the first cohort of students, I got some really choice quotes that I included in my last book, Minds Online: Teaching Effectively with Technology. And this semester, I actually have students reading some early drafts of the book I’m writing right now. And so it’s really been interwoven throughout my professional evolution over the last 10 years.

Rebecca: It’s pretty cool that you got to ultimately teach the class and it’s been going on for so long.

Michelle: Indeed, it is, indeed it is.

John: What do your students think about the role of technology in social media, in their lives, as well as in educational environment?

Michelle: Well, right from the get go, when I got to first design this class, and actually be sitting with a cohort of students every week, and bringing up a new topic, we divided it up into: there’s technologies for learning; there’s the effects of technology on aspects of thinking, like cognition, and so on; there’s several weeks on social media, which we’re right in the middle of right now. So there’s lots of different kind of articulation points where different students can come in with opinions. And so it does really cover that really broad area. So right from the beginning, I was so struck by the thoughtful and sometimes unexpected things that students would say… unexpected meaning kind of counter to what are some real stereotypes about… first of all, that all college students are a traditional age in this kind of lifestyle where you live in a dorm and party on the weekends. And I think most of us know that today’s college students do not fit that mold, and they’re not all that age. But, even students who are in this younger age bracket, to have them really say… like one of the early exercises we do in the course, I asked them to sort themselves on a continuum. We did it on a whiteboard this time via video conference, but in a physical classroom, they’d actually stand on different ends of this… place yourself physically on this continuum: Do you love technology, you want it everywhere, can’t imagine life without it… you hate it, you want to go low tech. And students are really spread across that spectrum. And so many of them have thought… they’ve said, “You know, I noticed I feel a certain way after I’m on Instagram for a certain amount of time,” or “I’ve tried electronic textbooks and I personally prefer paper.” …that’s actually consistent with some of the surveys that have been done with college students as well. So they are varied, they’re rich, and they are very counter to the stereotype that younger people just want technology everywhere in their lives.

Rebecca: What is one of the biggest misconceptions about technology that your students bring up in class that you address?

Michelle: Well, there’s a complex of sort of some interrelated ones that dial into my specialty area, which is cognitive psychology. So naturally, I noticed those really prominently myself. And so those ideas that using technology is going to reduce attention span, it’s reducing even your ability to think. And then there’s a sort of a related set of issues around what has been in the past a very controversial and headline dominating issue, which is the issue of taking notes by hand versus on a laptop computer in class. And that research, in particular, not to go through all of it, but, while the original study that sparked that debate was well designed, the interpretation of it has been just stretched until it screams. That study doesn’t talk about the distraction issue, there’s a lot of things that aren’t addressed in it. But students have come away, they’ve heard this kind of very superficial version of that, by which laptops are bad, and they also have kind of picked up a folk belief that if you handwrite something, it sort of drives it into your memory automatically. And it does not work that way. In fact, if you read the original study, one of the things that they say is that in as much as laptop note taking can be less memorable, whatever you’re taking notes on, It’s because you’re less likely to paraphrase, synthesize, and compress down what you’re hearing. And yet we have other people, they’ve heard these people in the culture say, “Oh, well, if you want to remember something, sit down and copy it, get out that pen and paper,” and that’s not really an effective study strategy. So they’re a little surprised and they say, “Oh, wait a minute, there’s some nuance to that study, and maybe some others that didn’t replicate it.” That study wasn’t talking about distraction on a laptop, it was just strictly speaking about this one aspect of how memory encoding works. Attention span… I probably talked about it on an earlier podcast… This is not a concept that attention scientists usually use. And so right off the bat, that’s a little suspect. And there’s not really good solid evidence that fundamentally, attention is changing. So they’ve absorbed some of those things. And so they’re really delighted to really dig more into those. So I might assign them an editorial or something that ran in a popular magazine or a blog. And then we look at the original research they’re talking about, and we pick up on the discrepancies. It’s not that Mueller and Oppenheimer was badly designed, it’s just they were looking at some effects that don’t always hold up with replication. And that speaks to this idea that the effect size is maybe not that large. Not that, again, anything was wrong with their data, it’s just you have this now you see it, now you don’t quality with some of these effects. And that kind of tells you that maybe this isn’t the hugely overriding consideration. And subsequent studies too have talked about this storage function of notes. It’s neat to think that you remember as a function of note taking without having to go back and study. But in reality, that’s what we do with notes, we go back and we study them. And so here’s this big elephant in the room like, well, are they taking good notes? And if they’re not taking good notes that capture key points, that they are going to want to go back and study actively, then picking up a little bit here or there because it was more memorable during note taking is not as big an issue. So that’s a big like, “Okay, what have you heard? Let’s look at the original research.”

Rebecca: Having the opportunity to talk about these things with students is exciting. And I’m sure the students are really into it, because it connects to their direct lives. And diving into the research makes a lot of sense in the context in which you’re teaching your course within psychology. So it seems like a natural fit there. How might other academic fields adopt some of the ability to talk about these things in their own classes where maybe cognitive science is not or psychology is not, the fundamental underpinning of what they’re doing?

Michelle: That’s, I think, something that I think is really exciting and why I am so excited to be able to share with your Tea for Teaching audience is I’ve really come to believe that that maybe there is something that is more versatile here beyond just the psychology frame and just a senior capstone in psychology. And I think that this is where faculty creativity can come in. I think the fundamental things that I think are so promising… Well, first of all, this is just a topic that is really under discussed, and it’s under discussed in a serious way. It’s not like students have not ever heard anybody critique technology. They’ve heard that. They’ve heard, “Oh, it messes with their sleep that it messes with their social relationships.” They’ve heard a lot of this, but it’s kind of swept under the rug in a way or even treated as “what serious person would ever think about these sorts of things?” So, that said, this is something that, and it’s something that students are doing all the time, even pre-pandemic. Most students do use technology of one form or another and are on one or more social media platforms. And so this is in and out of their lives all day long. So I can only think that there are critical frames and key concepts within a variety of disciplines that could map onto this, even if a faculty member doesn’t have the opportunity, or the interest, to say develop a whole course. Well, perhaps this could be a vehicle for discussing, for example, experimental design. How do you set up a study to really get at things like “What are the impacts of heavy cellphone use?” You do have certain individuals who self select to use technology in a particular way. And that’s something that you see crop up again and again in the research literature. Or if we’re talking about our own personal relationships, classes that have a focus on health can perhaps use one of these sub areas as a springboard for discussion. And so this is just really what I found, is that students who might otherwise be very quiet or, when things are framed in a purely very divorced from reality academic way, they may hang back, but who doesn’t get hooked into a discussion of some of the impacts of technology on our life. So I think it can be a vehicle for those things. And I think that it might be a little bit of a stretch in, say, a physical sciences class where we’re really discussing empirical context. But even there, it can be folded into discussions of effective studying very well, as long as we don’t just have that, again, very superficial tech’s bad, just get rid of it all and do everything on note cards. There’s a lot more to it than that.

John: Students are going to be interacting with technology, not only in their classes, but in their future careers. So having them think about those issues can be a really useful thing to learn, no matter what discipline they’re studying,

Rebecca: It seems like a good hook. It’s something that everyone can relate to, in some context. I was doing an exercise in my own class not too long ago about storytelling, and how brands present stories around what they’re presenting to people. And I use Spotify and Pandora as the examples. I’ve never seen a class so excited, [LAUGHTER] because it was talking about this technology platform that they can connect to. So I can imagine, when you bring up social media or other things that they feel really connected to, it immediately is a hook to talk about anything more complex.

Michelle: Absolutely. And that’s precisely the kind of dynamic that I’ve seen. And if I could throw out a kind of a discipline-specific example, there’s a concept that I really started weaving in more of over the last few iterations of the class. And this is a concept from psychological sciences research and quantitative analysis that really can be very slippery. But it’s a big, big part of contemporary ways that we analyze data. And it’s a concept of mediators and moderators. And so it’s jargony… and essentially mediators, when you have a correlation between two things, and you want to know, does A cause B? Or is there something else in the middle does A cause B causes C, and we have these great techniques for untangling those relationships. And moderators, on the other hand, is the relationship or is the correlation stronger in the presence of a particular variable or for, say, a particular group of people than others? And so yeah, you read that in a textbook and you go, “oh….” and yet, it’s one of the things that we really… I mean, experimental design, and how we can interpret our data is just radically more sophisticated when we can just not say, “Well, these two things happen together, but for whom is this relationship stronger,” and so on? So there are a lot of studies on the effects of technology that have one or more of these involved. And yeah, it just clicks for students when they see it play out in this relatable domain. So, for example, we have a study that I incorporate really early in the course. It’s got a word in the title, “Technoference” in relationships. So it’s a study of your perception that your partner in an intimate relationship uses their phone…. and when you’re talking to them… [LAUGHTER I think, will have a little bit of recognition if we’re in a relationship. That’s part of contemporary relationships, right? And they look at overall well being and how that relates to being in a relationship where your partner’s on the phone all the time. Now, it’s not a perfect study. And that’s part of what we look at. It was only among women who were in opposite sex relationships, and there’s a lot of self report and all that stuff. But you can say that “Okay, now they have a mediator. It’s not that the phone itself is degrading your life’s wellbeing but here’s this chain of causality of when your partner’s using your phone all the time when you’re talking, then you’re not as happy in your relationship. There’s conflict and then your overall wellbeing in your life goes down.” And then, in that context, you go, “Oh, Okay, I get it. Here’s what a mediator is.” And then we can talk about moderators, we can say, “Well, what about individuals who are in same sex relationships? What about men? What about couples who have been together for 25 years versus those who just got together six months ago?” Oh, okay. Now we understand moderators. So yeah, similar to you, Rebecca, I’m just saying, once you bring in some of these things, is not just dropping in sort of pop culture, it’s really taking a substantive look at these things. But yeah, then you springboard into concepts that are otherwise just really abstract.

Rebecca: Do you have some examples of things about learning related to technology that we might be able to slip into any discipline’s classes? …some of the stuff about attention, or good study strategies, or anything that’s maybe mediated through technology, but would relate to anybody.

Michelle: Definitely, the relationship between attention and memory and learning. Now, like I always say, when I’m talking about these topics, memory is not the only important aspect of learning. Learning is not all about memorization. But we now know that when you remember more, we have a broader knowledge base in an area, you’re better able to think critically and think in some sophisticated ways in that area. So that’s all good stuff. So that’s one piece of it. And in order to acquire any new memories, pretty much, for practical purposes, you have to be paying attention. And this is what devices and technologies have been so well engineered at this point to take away from us. So yeah, when you talk about a life skill, you’re going to need this for the rest of your life, no matter what you do. We have to think about, alright, how do we kind of shepherd and be stewards of our own attention. And I think, from a teaching perspective, too, it’s not that we have to constantly entertain students to grab their attention back from whatever it’s wandering off on, or similar that we just have to stand up there and be like, “Well, you have to pay attention… unbroken for an hour and 15 minutes… and all violations will be punished.” There’s different paths between those, but just to share with students that “Yeah, using phones is probably not changing the way our attentional systems work.” They work the way they have for many, many millennia. However, there’s a lot more competition for that now. So having them think about what are their strategies going to be. For some students, they come up with very creative cold turkey types of situations or types of strategies. I had one student say that I put my phone in a dropbox outside at night when I’m studying, and if I want to use it I have to go out there, which may not seem like a big deal, but in Flagstaff, it could easily be three degrees Fahrenheit and ice falling out of the sky, it’s cold out here. So we have students who say, “Well, you know what, I’m gonna be a little bit more subtle. I’m going to use one browser for my classwork and one browser for fun and social media.” And it’s just a little subtle cue that kind of tells you, “okay, we’re in work mode, or we’re not in work mode.” It’s not as much prescribing the answers as getting students themselves involved in saying, “Well, here’s how I’m going to manage this.” So those are some of the things that we would share. And when it comes to learning strategies at work, I’m always going to be evangelizing retrieval practice in one form or another. Lots of ways that that can look… everything from a Kahoot! quiz to sitting and talking with your roommate to try to bat back and forth what you remembered. Lots of different things you can do but, it shows too, there’s a link between you have to put in some active effort for your brain to pick up on that information and store it away in memory if it’s going to. So yeah, there’s sort of a complex of interrelated principles and take homes, there.

Rebecca: The one thing that I was immediately thinking about when you said about phones being really good at taking away your attention. I immediately thought as a designer, what a great example of how to get someone’s attention? [LAUGHTER]

Michelle: Yeah.

Rebecca: …not only to think about how to manage attention and think about what you’re paying attention to, but how do designers actually manipulate that? [LAUGHTER]

Michelle: See… perfect. There’s a cross-disciplinary connection.

John: The importance of attention is a topic that I think all students recognize is a problem. But I don’t think they fully understand quite how much of a problem it can be. Or at least my perception is there’s still a lot of misperceptions about the ability of students to multitask effectively. And I know that’s something that you address a bit in your classes.

Michelle: I do. And a related project that we’ve discussed on some other podcasts is the Attention matters project and I’m happy to report that project is still just perking along like crazy. We still have lots of faculty who are involved with it. So to kind of give a little background on it. Attention Matters was a concept that came out of a great conversation I had with my very smart and dedicated colleague, John Doherty, who’s an instructional designer and a librarian here at Northern Arizona University. And I had been going around and trying to teach a little, almost guest lecture, roadshow for interested faculty to spread these ideas to students of how to study effectively and how to have a plan for not getting distracted in the middle of class and stuff like that. And we talked about it. And we put together an online module that can serve so many more students. This semester, I have several really smart research assistants, undergraduate research assistants, who are in this module, moderating it and helping it run. And for those who know what MOOCs are (massive open online courses), it’s a little bit like that, except it’s specific to our institution. And so, in this, it’s a way of reaching out to students, they oftentimes will earn a little bit of extra credit in their classes for faculty who really want to spread these ideas to their students. They work through these modules that do touch on some of these key ideas about… as far as multitasking, we tend to be very overconfident. You can’t learn by osmosis, you do need that directed attention. Instead of highlighting and passively hoping things soak in, get in there and do retrieval practice. There’s also a little piece of Attention Matters, by the way, that talks about driving safety, which was not really something we set out to do. But I feel like it’s, again, a relatable everyday example that people can say, “Oh, my gosh, I was in a bike accident by a distracted driver,” or “I’m very careful about this.” And students are very adamant, and have strong views that do funnel back to that idea of: if you let it, devices and distraction of all kind can really take over and create some serious consequences. So, that’s yet another way that we’ve been working to bring these ideas to students throughout the years. And yet another thing that’s given us a fascinating window into what students are already doing to cope with these things, and some of their unexpected attitudes and ideas about them.

Rebecca: The thing that a lot of folks are doing is they’re teaching remotely or trying to jazz things up in synchronous online classes is trying to play with the idea of gamification in their classes, which certainly comes from technology, and often from video games and then some experience around that. Can you talk a little bit about how faculty might use gamification in their classes? Or also how that works on students?

Michelle: Yeah, games and gamification has been such a topic for so long in how can we use technology for education? I know it’s funny, when I was doing research for Minds online, I actually went to a Musee Mecanique in San Francisco, as a sort of a background research. It’s this amazing Museum, that’s just whatever the technologies of the time were, and it goes back like 100 years, all these different games, physical games you can play there.

Rebecca: It’s a cool Museum,

Michelle: Oh, you’ve been there.

Rebecca: Yeah.

Michelle: Oh, my gosh.

John: I was too.

Michelle: People have used photography in games and gamification. They’ve used all these different ways of using tech to play. So this is not a modern concept. And so we’ve seen lots of attempts throughout the years to also harness it for learning… some more successful than others. It’s such a deep theme in those connections between mind-brain-learning technology. And so students, here too, they get pretty excited about it. And that’s a good thing for faculty who are looking to use games and gamification. Now it’s another where I think drilling a little bit below the surface is really beneficial. It’s pretty clear to me, from the research and literature so far, that what makes games effective, and what makes them so compelling, you know, elicits the time, effort and attention that you need for learning, it’s not the superficial stuff about the experiences, not the music, and it’s not just calling it a game. It’s not necessarily tacking points onto something, although points and scorekeeping is usually a part of most compelling games, for sure. But there’s deeper things about getting really rapid feedback, there’s the opportunity for friendly competition. And that’s something that I’ve really seen this year, because I’ve also been using quite a few quizzes and polls and things like that in my courses, too, that are remote, is that you don’t have to attach a grade to the game to get some students really into the idea of competition, while other students, there, it’s more anxiety provoking, or it’s just too much because they’re already in so many high-stakes competitive exams, where they can play for fun. And so those are some of the aspects that are important when people are thinking about selecting a game, setting up a game, bringing gamification in some way. It doesn’t have to all be cheesy, let’s make everything look like a video game. But really, that idea too, that mistakes are part of it. While we’re playing a Kahoot and you get an answer wrong, whatever, we’re doing something else in five seconds, and it’s not a big issue like a test question is. So there’s definitely that. And I would say, too, that students here as well, they can be a great source for insight. So talk to your students. Say “What aspects of this game are more appealing? less appealing?” and so on. And games and game culture too, this is something that I really get a sense that they’ve never had a serious, let alone academic, conversation about the role of gaming in their lives. Yet for many students, that’s an important part of their identity. It’s what they do to relax. It’s what they do to socialize now, quite frequently, especially with distancing happening. So, as weird as it might sound, let’s take games seriously. Let’s take games seriously as an important aspect of students’ lives. Let’s take it seriously as a road to learning. And let’s just keep exploring that because the more research that gets done, the more effective and beneficial features we find associated with games.

John: And the most popular games are those that students can work through. And no matter what their prior knowledge with that type of game, as you said, provides them feedback. And that feedback is targeted so that they can use that to improve and the level of the games are set so that it’s neither so challenging that they give up and get discouraged, but not so easy that they don’t have the sense of challenge. And that seems like a really good way of perhaps thinking about how we should design our classes in general, whether we include explicit gamification aspects or not, creating an environment that encourages students to actively want to engage with the material, and where they can see progress and see how they’re advancing. That is, in general, something that I think is a really important thing for us to contemplate at least in course design.

Michelle: Agree 100%, agree 100%. And that’s exactly what makes games compelling. What is about social media that makes people return to it again, and again, and again, hundreds of times in a day? And what features can we extract and adapt in the service of learning?

Rebecca: One of the things we talked about with Ken Bain last week was an example about the arts and how that might change someone’s thinking… an experience with a piece of artwork. So, I used that kind of example, to inspire a little activity with my students this morning. And I asked them, “Can you talk about a piece of artwork that has influenced your thinking?” And I gave them some categories. And I’m teaching an interaction motion design class, but I included visual art, but games were one of my categories. And some of the students put some really interesting examples about how certain games have gotten them to really contemplate interesting ethical questions, relationship questions, really interesting stuff. And they wrote really thoughtful responses. I had them basically write the name of the game and just a sentence about how it impacted their thinking. But there were some really thoughtful responses. And it was really almost surprising to me how deep some of those quick summaries of their experiences had been with games.

Michelle: Yeah, that’s perfect. And without the conversation, you wouldn’t have that window.

John: For many years, we’ve all heard lots of arguments from faculty about whether technology should be or should not be used in classes. The pandemic, to a large extent, has shut those down completely. And that’s been, for many of us, quite a bit of a relief not to have to deal with those arguments all the time. However, as we begin to move back into a more traditional onsite teaching environment where more instruction is taking place in regular classrooms again, what are some of the things that people may have learned about interacting with technology effectively during the pandemic, that may perhaps lead to improvements in how we teach our classes regularly?

Michelle: That is such a meaty question, and I think it’s one we’re going to see so much just rapid development of reactions. it ties into the whole question right now of what does instruction look like post-pandemic or whatever the next stage of the pandemic is? But yeah, what a good time to think about this. And you know, I can look at it too through the lens of faculty experience, I was kind of fortunate to have had my Zoom baptism completely by accident earlier in spring of 2020. Because I had set up this idea of having a lot of guest speakers in one class, and I got a huge response, which is wonderful, but I needed to bring them in. And I had always kind of said, “Well, if I’m going to Zoom, I’m going to kind of sidestep that. I’m going to let somebody else drive.” And I had to get over that really fast. And so I do think that it illustrates the value of some targeted, not totally strategically planned, practice with technology tools. And that’s just the kind of bedrock cognitive processes that, when you have something like being able to just run Zoom, or Collaborate or something like that, or have an online poll, your ability to do that while monitoring a classroom or answering questions, you got to have the practice in first, and our students are the same way. So we can think about, alright, whatever we’re going to have students interacting with or using or if it’s us that are using something, having that practice upfront and expecting that, once we’re on the other side of the learning curve, it looks very, very different. So that is one big part of it. On a much more conceptual or abstract level. I think that, this whole year, we’ve really needed to look at the students and their goals and why they’re there in the class in the first place, wnd why are they taking the course. That’s something I’ve written about in some of the shorter articles I’ve put out this year. I think the pandemic teaching was distinctive for a lot of reasons, but one of them is that you just can’t keep persisting with “Okay, I’m the learning cop here and I’m going to make sure everybody does things because I’m watching you.” At the end of the day. I found if my students… I hope they’re not, but yeah, they are in Zoom, they could be doing other things… they may be minimally attentive, and that is not good for their learning. And I do a lot of things to have a lot of different shifts in gears to bring in gamification. I’ve done a lot of things to do that. But ultimately, if the student wants to check out, they can check out to an extent. And I’m not saying I’m okay with that, but I think that we are going to be meeting students much more in the middle, instead of having a more adversarial relationship to their learning, I’m here to enforce what you have to know. I mean, we have to collaborate to have something like remote teaching the way we’re doing it, to have that work at all, there has to be more of a collaborative approach to it. So I know that that’s a very top-level conceptual type of answer. But I think that in a lot of things, we’re going to be saying, “Well, you know, what, if this is something that helps some students, and if I’ve talked to students about why they’re here, and they’re purpose driven, ‘I am here to actually learn and take something from this class because I need it for the next class.’” Well, that’s a great basis to springboard off of, instead of “how do I write the policy in my syllabus that will prevent any kind of behavior I see as undesirable.” And you know, so many people were already moving away from that, which I think is incredibly fortunate given what we’ve been through in the last year. But this may be, if not a tipping point, something else that pushes us more in that direction of saying, “Well, what are the policies there to do?” Yes, students have to pay attention to learn. And that is very, very clear during remote pandemic teaching, as well as everywhere else. But let’s maybe take some different approaches and have a different philosophy of how we get there.

Rebecca: One of the things that I also hear you hinting at Michelle is that during the pandemic, we’ve all had a lot to manage, we’ve had a lot of cognitive load. And so we have to prioritize, and we have to decide what’s going to win our attention. And so students have the same problem all the time, just like we have the same problem all the time, we’re just more aware of it now. They have multiple classes to balance, they might have family concerns, they might have jobs, and at some point, they’re making choices about what they’re going to attend to, and what they can’t attend to. And I think sometimes we always hope and wish that they’re attending to whatever we’re putting out in front of them. But that might not be the best choice for them at a given moment, based on the other things that are going on in our lives. And we just often don’t think of our students in that kind of holistic point of view.

Michelle: Oh, absolutely. That’s such an eloquent example of this way of thinking, and the things that we have learned and the shift in mindset that we may be on the cusp of. And that’s another thing that really underlies the approach to talking to students about technology that I’ve really come to adopt, which is the same-side instead of opposite-sides stance. Like you said, we do struggle with some common things. I’m caricaturing a little bit, but I think we’re playing off of an older mindset where it’s us, we’re were older, we’re in this position of authority, and here’s how we like to do things. And here’s this young generation, and they think, very alien to us, and they want to do something else, and we’re going to make them come over to our side… saying, look, we all get distracted. In class, I’m frequently saying, “Well, yeah, here’s something unpleasant that happened to me on social media,” even if I don’t tell them all the details. [LAUGHTER] The point is, yeah, I get misunderstandings and hurt feelings on social media, too. I end up in the social comparison that tends to be so toxic on places like Instagram. I get really, really distracted and sidetracked because I’m using the same computer for 20 different things all at once. And so let’s work together to see how we can address those challenges. And yeah, so I think that what you’re describing is, I think, a very healthy way forward.

John: Now that faculty have had a chance to get more insight into students lives, perhaps now faculty will be more understanding of those things in the future, because the classroom environment is somewhat separated from all that it was much easier to ignore those things and maybe faculty will be more likely to treat students as human beings, perhaps in the future.

Rebecca: Are you implying that the classroom is real life?

John: Well, maybe it may more closely resemble that as we move back into more traditional classroom settings.

Michelle: Yes, and I’m all for that.

John: We always end with the question, and it’s particularly relevant now, “What’s next?”

Michelle: As you mentioned at the top of our interview together, I am in the very final stages of completing the Remembering and Forgetting in the Age of Technology book. So I’m really excited to having that book be coming out in the not too distant future. And I’m really throwing myself into a brand new professional role, which is as the Co-editor of the Teaching and Learning Series with West Virginia University Press. Now, this series has just drawn so many dynamic thinkers with so many practical and also evidence-based ideas that we can all use in teaching and learning and so it was a tremendous honor to be invited to take that role on and I’ll be working with the other editor of the series who founded the series and launched it all, Dr. James Lang, who has just been tremendously influential in the area of bringing evidence-based effective pedagogical strategies to so many people in higher education. He’s been this tremendous leader in that area. His writing is also amazing. So what an honor to get to work with him and with West Virginia University Press. Stepping into that role has taken up a lot and it’s been wonderful already. So that is, for the most part, what’s next for me.

John: And I think we could say the same about your writing based on your earlier book, as well as recent comments that Jim Lang made on Twitter about how much he enjoyed the clarity of your writing and your exposition in this new book and how much he’s looking forward to that being released.

Michelle: Oh, thank you, that’s so nice to say and being able to teach students and to talk to students for so many years about these issues was the inspiration that gave me ideas to work with. So, it all comes around.

Rebecca: Well, thanks, as always for joining us, Michelle, and sharing some of your insights and some of the work that you’ve been doing.

Michelle: Oh, my pleasure. Thank you.

John: Thank you, Michelle. And we’re looking forward to talking to you about this book as it gets closer to coming out.

Michelle: Absolutely.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

183. Student Workload

College students throughout the country have reported substantial increases in their workload during the 2020-21 academic year.  Few faculty members, though, intentionally increased student workloads during this challenging year. In this episode,  Dr. Betsy Barre joins us to explore some reasons for student perceptions of increased workload.

Betsy is the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator.

Show Notes

Transcript

John: College students throughout the country have reported substantial increases in their workload during the 2020-21 academic year. Few faculty members, though, intentionally increased student workloads during this challenging year. In this episode, we explore some reasons for student perceptions of increased workload.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist&hellip

John: &hellipand Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Betsy Barre. Betsy is the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator. Welcome back, Betsy.

Betsy: Thanks. It’s great to be back.

John: It’s great to talk to you again. Our teas today are:

Betsy: So, I’m not drinking tea. I’m having many cups of coffee today. [LAUGHTER]

Rebecca: Well, it’s still warm&hellip

Betsy: Yes, that’s right.

Rebecca: &hellipstill warm, and still caffeinated. [LAUGHTER] I’m drinking Irish breakfast today.

John: And I have ginger peach green tea.

Betsy: Nice.

Rebecca: &hellipan old favorite. So we’ve invited here today to talk about your recent blog post that addresses the impact of pandemic instruction on student workload. Can you talk a little bit about how the pandemic has affected student perceptions of their workload?

Betsy: Yeah, sure. So this issue has cropped up for many of us. I’m sure anyone who’s listening to this podcast has&hellip maybe in the spring, but particularly in the fall… and I think that’s really interesting to that in fall it became an even bigger issue than it was last spring&hellip that we started to hear from students in our online courses, and in our blended courses&hellip not just online&hellip that workload was overwhelming, perhaps even double. And we heard it at Wake Forest. We started hearing it anecdotally. And then I would talk to my colleagues at other institutions who, of their own initiative would bring it up, that they had heard it anecdotally as well, we saw on Twitter folks talking about this. And then we at Wake Forest did an all student survey where we didn’t ask about workload&hellip we probably should have. But it was the number one thing that came up in their open ended comments when we coded those. And so it just reinforced this idea that clearly this is a universal challenge. And it was a challenge across our schools to, so it wasn’t just our undergraduate students, We were hearing in our Divinity School and our law school and our business school. And so something was going on. And it was really intriguing to me, because clearly students felt like the workload was overwhelming. But, and this is what we’ve all said. It’s not as if all of us just sat down and said, “We want to give students a lot more work this semester.” So I was fascinated by it, talked to a bunch of people about it, was thinking about it. I know you all have been thinking about it, and just decided to write some of my thoughts in a blog post. One of the great things in the response to that blog post is lots of folks have come up with other ideas that I think are just as plausible too.

Rebecca: Do you think faculty believe that they’re giving more work to their students?

Betsy: That’s actually a really interesting question, because that sort of premise of my blog post is that, and this was Jody Greene said “No one sat down to give more work to students.” But since I’ve written it and talked to some faculty, there are some faculty who are like, “Yeah, maybe I did, maybe I did give a little bit too much work.” And that’s worth noting. But there are just as many faculty, maybe more faculty who say “Actually I have given less work this semester, and I’ve tried to dial it back and lower the stakes than I have in the past.” And so the fact that there’s that large body of faculty that think they’re doing the opposite, and then the student perception is something different. It’s really interesting.

Rebecca: One thing that you just said, Betsy, about the lower stakes piece, raises an interesting question, because a lot of professional development about going online and using effective teaching practices talks a lot about low-stakes assignments and the ability to check in on things more often. But maybe they’re smaller assignments. Do you think that’s happening more?

Betsy: Well, I think it’s a good thing that it is. And my guess is, that’s part of what’s causing the problem or the challenge&hellip maybe it’s not a problem, but just is causing this sort of disconnect… is that our faculty, particularly many of our institutions, in the summer did a lot of professional development around good online teaching practices, and just good teaching practices in general. And also really emphasized&hellip at least at Wake Forest&hellip we really emphasize this is a pandemic, our students are struggling, let’s lower the stakes on things, let’s be understanding. And so one way to do that is by having smaller low-stakes assignments, so instead of a big midterm, you have multiple weekly check-ins. But of course, our students can interpret that as, “more work,” because if you’re just counting work by counting the number of assignments, then it is, by definition, more work. So, instead of one midterm, you now have five short assignments, that’s five times the amount of work. And so instead of counting in terms of how much time the assignments take, they could be counting and just the overwhelming number of assignments seems like more work. And I think that’s what’s going on, or at least part of what’s going on. And I’ve said to some people that this is actually a good sign that change did happen over the summer, because we didn’t hear as much about this in the spring. People kept their one midterm and their final&hellip at least I didn’t hear about it as much, maybe you two did, but I didn’t hear as much about it in the spring. But then they redesigned their courses in the fall. And the fact that we’re all hearing about this suggests that people actually did things differently. Now, again, it still could be better, but that’s kind of a good sign to me. Now, the question is, how do you dial that back? And how do we communicate with students about it&hellip all really complex, but I do think it’s that breaking big assignments into smaller assignments is part of a contributing factor here.

John: . And we know that students tend to do a lot of cramming, they tend to do mass practice, but we know that spaced practice is more helpful and that we know the benefits of retrieval practice. And that’s something I think that most faculty development centers emphasized with faculty. And I know at our campus, we had more faculty participate than we’ve ever seen. We had more people participate in professional development workshops than we generally see over a four- or five-year period. For the people who were resistant to professional development in the past, they were learning about the benefits of retrieval practice and space practiced, and learning about the benefits of using low- stakes exams, as you were just talking about, and I agree that that’s a good thing. But we know that the practices that students use to study tend to be mass practice, they tend to do repeated rereading, and now they’re being asked to retrieve information. And we know that students believe that that’s less effective, and it’s certainly more work for students.

Betsy: So your point about retrieval practice, and we know students believe it’s less effective to be engaging in this continual retrieval practice, I think is really interesting. And I think that’s what we’re seeing when our students say, “We have more busy work.” So it’s not just that there’s more work, but that’s actually more busy work. And part of what’s going on there is that they think that that practice that they’re engaged in is not valuable, if you are giving assignments that are about practice. And as students see it as busy work, that’s part of us communicating the value of this work, and helping our students understand how they actually learn, and how it will help them on the later exams, I think is really important. That’s not the only challenge. I think busy work isn’t the only kind of challenge. It’s also, I think, for those of us in the humanities, I think what we’re seeing is that the new tools we have available to us make it easier for us to hold our students accountable for doing all the reading, when typically, they wouldn’t do all the reading. And typically students wouldn’t say it’s busy work, but there’s more reading that they have to do than they ever had to do before. And so that’s one hypothesis as well. But I think another point about the busy work and the retrieval practice, moving one exam to 10 short assignments is, and I talk about this in the post as well, is that there is a sense in which that could be adding to your work, in that they have to keep track of it all. And I think our students are not used to having to keep track of so many assignments. So typically, as a faculty developer leading a teaching center, I may have a faculty member come to me that wants to redesign their course. And I make all these suggestions, and they do it and it’s fine. And the reason it’s fine is because, yes, it’s a little bit more work and a little bit more stressful for the students. But it’s only one course. But I think what we saw is that all of a sudden, our students were moving from five courses where there were three assignments to five courses where there are 15 to 20 assignments or more. And that was even more compounding the exponential growth that they felt. So I teach with lots of small assignments, I always have. Students would sometimes say this is more work than in a typical class, but they weren’t upset about it. They didn’t feel overwhelmed by it. That’s because again, it wasn’t five of their courses that were doing it. So it is a really interesting question of when we go back post-pandemic, do we want all of our courses to work this way? And how do we help our students readjust to this is the new workload? or this is going to be the new experience of the new workload? Or do we not want to do that? And I think that’s an interesting conversation for all of us to have moving forward.

Rebecca: I know one of the things that I’ve had in conversations with students, just anecdotally, but also in some of the formal research that I’ve been working on related to students with disabilities, is the time management piece and just trying to manage and organize all the moving parts that are on all these different platforms is complex, but also that moving with more materials online has resulted in more reading and writing&hellip

Betsy: interesting.

Rebecca: &helliprather than other modalities that we might typically use in a face-to-face class like face-to-face conversation, which to them seems really much more time consuming. And it may actually be more time consuming, especially if you have a particular kind of disability.

Betsy: Right. So there are a couple of things to say there. So I did say one of the things I noticed is when you read student concerns about this, they will often say things like “It took me this much time to do a discussion post.” And that’s, I think, really revealing for all of us to understand. We often think, “Okay, the discussion posts are going to take the place of the discussion in class.” But right now, I’m just talking to you two, and I’m not thinking very hard about what I’m saying. And in fact, if you created a transcript of this, which you guys probably will, I’ll be embarrassed to read it, because I don’t think it’s as coherent as I want it to be. And if I were writing a discussion post, I would think very carefully about how I formulate my thoughts and my arguments, and even proofread. And it’s gonna take a lot more time, if I’m actually writing it out. And I think that’s really important for us to acknowledge that discussion posts and a discussion are not a one-to-one replacement. Or if we want it to be a one-to-one replacement, then we need to tell our students, we expect you to treat it as if you’re not actually writing something that’s meant to be thoughtful, we just want to hear your opinions about this. So that’s one piece. But then in terms of your point about disabilities, I think it’s really interesting in that all the best practices for Universal Design for Learning, we can revisit, and I didn’t talk about this in my post, but I should have, so thank you, Rebecca, for sharing this wrrinkle. Because I think it’s an important part of it is that giving students options for how they can do this work will also empower them to do things that they think are most efficient for their time. So if they can do a VoiceThread or make a video, or one of the activities that some of our faculty have found very successful as an asynchronous replacement for discussion, is to just put students in groups and tell them, in your own time, you get together, have a Zoom discussion about the material, record it and send it to me ,that you’ve had that discussion. So they actually have a discussion. It’s just sort of asynchronously done. But in general, giving students options, it’s not going to solve every problem, but it does empower them to have choice, because there will be some students who prefer to write than to speak. But there may be something like “I’m tired of writing, I want to actually just speak.” And then in terms of the material, I think there was this recent meta analysis that just came out like last week about video versus text, which was really interesting. And as a humanist, I’m sad to see this, but it’s not surprising that sometimes video can be better for student learning than a text can be. Because I often think, “Oh, I’ll just give them something to read, and that will be the replacement for a lecture.” But maybe sometimes there’s a way in which they’d rather watch somebody talk about that material, rather than read about it.

Rebecca: Or by extension, just listen to the material, like in a podcast or something.

Betsy: Yeah, podcasts are a great opportunity. And we’ve heard students say, when I’m walking around campus, or when I’m working out, and it allows them again, to expand their schedule where they have more time to do things and no screen time, which is something they really appreciate because there’s so much Zoom fatigue, that being able to listen to something where they don’t read online and then have to watch online, they can just listen to it is a real relief for them. Absolutely. Unsurprising you all like podcasts.

Rebecca: Anything that gets us off the screen, actually, is something that I work a lot to do with my design students, because whether it’s a pandemic or not, we spent a lot of time on the screen.

Betsy: Yeah, fair enough. So you’re an expert at this. Yeah. You’ve thought about this. That’s great. Yeah, for sure and I think we need to think about that more.

John: But I know even for people who are teaching asynchronously before, some people have started using new tools. On our campus, for example, people who used to give students readings as a basis for discussions now are having students use hypothesis for the discussions, which means students actually have to open the reading [LAUGHTER] and actually respond to the text, which can take a lot more time than just skimming over the abstract and responding to it. And similarly, I’ve been posting videos for 20 some years in my classes, but now I’m doing it where there’s questions embedded in it, which means they actually have to watch them now for a small portion of their grade. So I think some of the tools that people are using may provide more learning, may provide more engagement, but also is going to take a lot more time than how people use them before. And you noted in your blog post that many students would be able to get by and coast to get the grade they want without doing a lot of the things faculty assume that they did. [LAUGHTER] But again, I’m not sure that’s a bad thing. But it does require more time on average.

Betsy: Yeah, it’s a complicated question. It’s a self report, so it could be even lower than this, but just general self report on how much time students spend each week studying, it’s about 15 hours a week, on average, prior to pandemic, and that is for a full-time student. So imagining 15 credit hours they’re studying, it’s one to one, and many faculty assume or hope that it’s more than one to one. [LAUGHTER] But students are very strategic, they’re learning an important skill and figuring out what does need to get done and what doesn’t need to get done to be able to be successful in a course. And so certainly I appreciate that. But I think recognizing this disconnect is important because it helps us understand why faculty didn’t think they were giving more work, but students actually did have more work because faculty were mistakenly assuming that students were spending 30 hours a week studying when really they were only spending 15. And so being aware of that now helps us have a much more honest conversation about well, what do we expect the standards to be for students, and there are differences across different institutions and different programs. So our graduate professional programs are for folks who are working full time, have different sorts of informal expectations, I think, than others. And so it’s worth it for all of us to come together and to talk about that. But I will say I do think it’s just important to say&hellip I probably said this in the post… but we do know that the more time students spend on a task, the more they will learn. So it’s not just like we’re piling on the hours because we want to punish them or we think that’s just really what rigorous teaching is. It’s that actually we know you’ll learn more if you spend more time thinking about a text or practicing the problems, as you said, John, that this will help you learn more. So you obviously don’t want to expect so much that they can’t do other things they have to do in their life. So that’s the tension. I think my recommendation always be if you have to have a full-time job, you shouldn’t be a full time student, because that’s like too much work. So thinking about how do we calibrate the courses that students taketo how much time they’re actually able to put into it is really important. So yeah, I do think that that’s happening. It’s not the only thing. Again, I also think there are faculty who probably expect too much as well, because we’re not good at estimating how much time it takes for students to do things. I think Hypothesis is a great example. I use Hypothesis in my class, I love it. If you’re a humanist and you haven’t used, or if you have assigned readings and you haven’t used Hypothesis or Perusall go look it up and find it. It’s pretty amazing. But I think that remembering that, yes, it will make them read. So that’s extra time because they’re actually gonna have to read and they’re gonna have to read carefully enough to have good questions. [LAUGHTER] So they can’t skim it, as you said, John, but then all the time it takes to actually read everybody else’s comments, really remembering that and that’s where I as a newbie to online, that was like an aha moment for me when one of my colleagues who’s an expert in online teaching was like, “It’s not just the time it takes for them to write their own discussion posts, it’s also they have to read everybody else’s. There’s extra reading that’s involved.” It’s not just the text itself, but it’s also reading everybody else’s responses and so putting them in groups where they’re responding to fewer people or reading fewer people is a really useful tool. Again, I think probably all of these hypotheses are going on. And it’s worth us being honest about all of them, instead of saying, “Oh, it’s definitely the students,” or “it’s definitely the faculty,” it’s like we’re all in this together, and let’s figure out how we move forward.

John: A nice thing, perhaps, would be to give students information about how much time these tasks take. And it would be nice if there was a tool for that, [LAUGHTER] which I believe that you have created.

Betsy: So yes, we have a tool that actually we made pre-pandemic. But one thing I want to say, because a lot of people have used this tool, and I think sometimes people use it in ways that are asking you to do more than it was intended to do. And that it is very much an estimator. It is not meant to be a calculator, that is the exact amount of time that your students are going to spend on something. And it’s very broad. It was essentially just something that I was interested in creating as I was thinking about how much work I assign students in terms of reading and writing. And the original version of it is very much tilted towards reading and writing. So oftentimes, we hear from STEM folks like “What about problem sets?” And that’s and that’s just the Wild West in terms of how much time students spend on that, it’s much harder to get a handle on it, so it’s not there. But there are places in this estimator where you can add a new assignment that isn’t captured by reading and writing and just give your own estimates for how much time you think students will spend. And the main value of this estimator, I think, is that I found that many of my colleagues, myself included, are just not good at the head math required, we just keep adding these assignments, and we think we have a good sense, but literally sitting down and writing out like “Okay, they have to go to the library to get the source.” Well, it’s gonna take him some time to walk to the library and walk back&hellip like literally things like that, realizing how much time you’re asking your students, and then adding it up can be really valuable. And I would do it sometimes on the back of an envelope, but it was chaos. And so I thought, why can’t we just have a calculator that does that, So we have an old version of the calculator, we have a new version that my colleagues in online education at Wake Forest, Allen Brown, helped us work on to add in discussion posts and video lectures and other things so that it’s a little bit closer to what asynchronous online courses might involve. And it can be a tool for overall assessment, but also individual assignment assessment of like, how much time might it spend for them to do this type of reading or to do these types of videos. And if you disagree with what the estimator says, my favorite feature of the estimator is, you can manually adjust it. So you don’t have to get in arguments with us. Whatever your own assumptions are, you can go in there and put that in, and you’ll still be surprised with what the total amount is probably, at least I often am, that I’m giving more than I realized and I have to go back and make some hard choices. So hopefully, it’s a useful tool for everyone. But as John, you said, one of the best things about it is that allows us to better communicate with our students about what we’re expecting as well. And we’ve heard from so many students who have found it super helpful in the courses that have done this, both students who are struggling, but also students who are crazy overachievers, and who will spend 20 hours on a one-page paper. It’s a real relief to them. Even if they only spend four hours when they’re supposed to spend one, at least it’s four, and not 20. So it helps them manage their time as well.

Rebecca: One of the things that I’ve done, at least on longer term projects, that has worked really well for me and my students is having them keep a timesheet and asking them to divide out tasks. And I pose it to them so that we’re in the design field. So it’s to help them think about how they might price something in the future, so they know how long it takes them. So that’s how you get the buy in. But what it helps me do is see how long it takes them to do certain things. And realize it’s like, “Why did you spend this amount of time doing this thing that was really not important, as other thing was much more important?” And then you can coach the group on those sorts of things, which can be helpful. And along those same lines, one of the things that I run into, and this may fit more into the idea of problem sets or things like this is how much time students will try to problem solve a technical issue that they just aren’t problem solving in the right way at all. And so they could spend hours trying to do something that if they just asked a question… [LAUGHTER] &hellipit would have taken two minutes.

Betsy: Like ask for directions…. [LAUGHTER]

Rebecca: Yeah, so I’ve been reminding my students, especially since the fall, when we’ve been doing much more online,that, if you’re spending more than 15 minutes trying to solve this technical problem, A. take a break, you’re just going in circles, maybe come back and try again. But if you’re spending much more time than that, then that’s a good clue that you need to ask for help.

Betsy: That’s really smart. And really, I think, super helpful. And I think getting feedback from our students about how much time they’re spending is not just good at the individual level of coaching. It also is great formative feedback for adjusting our own expectations. Again, and it corrects the estimator, maybe you put it in the estimator, and has happened to me too. And I realized&hellip because one of the things about the estimator, it’s best about reading, usually, in terms of its reading estimates, but one of the central insights from the reading literature is that the difficulty of a text is just as much about students’ vocabulary as it is about the text itself. So I would guess “this is a pretty easy text for my second year students at Wake Forest.” And then if they’re all taking a lot longer, what I realized is that actually, I misjudged their familiarity with these concepts that would be in this book. That this book is actually harder than I thought it would be. So I need to up it in terms of the estimator to say “Actually, there are more new concepts than I realized that the students are engaging with and it’s going to take more time.” So asking the students is just as important as you communicating with them. It’s a two-way street for sure to get that formative feedback. I also think telling them about time management and struggling with time management. I’ve seen some really good strategies. I know our learning assistant center, who works with students, has some good counseling that they do with students about how do they create a master syllabus or kind of a calendar for when they’re going to do things. And I also saw somebody, I think, shared it on the POD listserv, but a strategy of creating a Google calendar with basically time slots for all of your activities in your course. And then students import it into their Google Calendar and move those around. So you would set it up like two hours for reading this text. And then they could move it in their calendar. And so that works for them. But they basically see the blocks of time that they need to set aside. And if they did that for every class, it would be even better, they could see “Oh, wow, this is 40 hours in a week, I need to set aside time to do this work.” And frankly, we should be doing that even before the pandemic. But we’re learning this lesson now of how to help our students manage time and due dates, and all of that, because it is a little bit more. And again, I also want to emphasize too, not just all the cognitive load of multiple assignments, but learning new tools also takes time. This is kind of your point about troubleshooting, Rebecca, like, if a student has never used the video function on Canvas, they may find themselves spending 45 minutes trying to get the video function to work, when that’s not in any of our calculations of their assignment. We’re assuming they’re just going to record the video and upload it. So being mindful of the time it takes them to learn a new tool in this scenario is also really important.

John: You mentioned the issue of reading tied to students prior knowledge and vocabulary. But that’s going to vary a lot across students. So I know a lot of people, when they include estimates from the calculator, will say this is an estimate of what this is, your mileage may vary and keep track of how long it takes you to do these things, and use that to adjust your future estimates of the time requirements for these tasks.

Betsy: That’s a nice idea too, to say you students adjust. So that’s really smart. I like that a lot. For sure, it varies across students. And especially, I mean, even thinking about students with disabilities is an even more interesting challenge. And there is an interesting question, I’ve had some good conversations about to what extent, if we’re putting that estimate… the average&hellip in the syllabus does that create problems for students who may be slower, they think that there’s a deficit. So you need to be thinking about how you frame it, I think is really important. And to be up front that saying it is expected and that is the normal course of things that we’ll all have different rates and this is a ballpark average. You can even put a range&hellip might be an idea too&hellip of ballparks there, but recognizing and saying it’s totally understandable that there’ll be jeans taking a different amount of time, because again, prior knowledge, not just ability, it’s all sorts of other things. How often have you read in the past? How often have you worked with technology in the past? Any of these things, they’re gonna make a difference.

Rebecca: One of the things that conversations about perceptions of workload lead me to is I wonder what the perceptions of learning are?

Betsy: Yeah, I think this is a great question. Because when we think about how students got “got by” in the past by doing less work, what they meant by “get by” was successfully complete the course and get the grade that they desired. If we actually ask them about how much they learn, I don’t know. I mean, that’s a really interesting question, would they say, “Oh, well, it takes more to get my A now&hellip” so that’s duplicating the workload&hellip “But, oh, by the way, I’m also learning more.” It’d be interesting to see. I mean, it depends on f the primary issue here is that students doing less work before and now they’re doing all the work we expected of them, then I think you would expect a lot more learning. But there could also be these issues of the pandemic, I’m in crisis, I can’t work as quickly. If those are the issues, or I’m overwhelmed by the multiple assignments, and I can’t keep track, then there may not be as much learning happening. So my guess is there’s probably equal levels of learning, it’s totally a guess. But in other words, that there are challenges to this moment that students learn less. But there are also things that we’re doing better than we have in the past that make up for that. But I hope that we get some good empirical data on some of this and think through it, because I do think that these strategies, while they are more work, are also probably likely to lead to deeper and lasting learning as well, if the students are able to do it. There’s also the challenge of students who just give up, and then get overwhelmed, and they’re just completely behind. And then they have no motivation to even do a little bit. And so we want to be mindful of that too. But if they’re able to keep up, I’m hopeful at least, that these things should, at least from the research, they should lead to more learning, but who knows.

John: In terms of student reaction, though, student perceptions of what’s most effective is often passive learning and repeated reading. Fluency illusion makes it seem that you’ve mastered the material without being confronted with some type of evidence that you really don’t know this stuff quite as well. And that all the techniques that we’re actively encouraging in teaching centers are giving students more feedback more regularly about what they know, and what they don’t know. And that doesn’t feel as good. And there was a study at Harvard about a year and a half or so ago, where they surveyed students on how they perceived their learning, relative to the actual learning gains they receive across both lecture-based classes and classes that relied on active learning. And there have been a lot of such studies where in general, the students believe that active learning is not as effective yet the learning gains tend to be significantly greater. So there is a bit of a disconnect between what students perceive as being effective and what actually is effective, which also can lead to that perception of busy work that you mentioned before?

Betsy: Yeah, I absolutely agree with that. I think, and this would be a whole other podcast you probably all have done all these podcasts thinking about this issue of student perceptions about learning. I think part of it is what they’re used to. There’s a lot of things that are going into helping all of us understand how we learn and what works and what doesn’t work. And so I think there is a hope for us to try to sort of bring them along with us, I guess I would say, I guess the valuable insight from the studies is that we shouldn’t take for granted that if students say they’re not learning, and if they aren’t learning, that we need to recognize that they may be. And so part of our job is to help them understand, with hopefully concrete evidence that we can show them, “Look, you’re actually learning here in significant ways” &hellipto help them understand why we are choosing these approaches. And it’s not just because we don’t want to teach or we’re lazy, or what are other stories people tell about active learning, moving forward. So I think part of the way we bring them along is to also acknowledge that sometimes there may be assignments that are not useful, and that there may be sometimes things that are overly burdensome in terms of time. And so instead of just always being “You’re wrong, students, you’re wrong here, let us tell you how it is,” to say, “Okay, let’s listen to our students” and say, “Actually, that assignment, it took more time than it was worth. And so we’re going to think creatively together about things that will work for you.” But also acknowledging that there’s a long literature on how people learn that should inform it, and not just perceptions that make a difference.

Rebecca: I think when I’ve even asked students about some of those things like “What do you wish you had more of?” &hellipthey do realize that when you have those little assignments to hold them accountable, and help them practice, I had students asking for more. In the fall, I had students saying “We had a few of those, those were helpful. We wish we had more of those.”

Betsy: Yeah, there’s no question. We saw that in our survey, too. And that’s the reality of anytime you do a study, it’s an average. On average, students think they’re learning less, but they’re always going to be students who, “Oh, I’m aware, I’ve seen this happening.” And they’ll be students who sort of totally missed the boat. But yeah, we saw that for sure. We saw students who appreciated the check-ins, but the number one thing that we saw from our students on various questions was that they wanted more opportunities to work with each other, which, normally, they don’t like that. And there’s literature, right? It’s like, “Oh, I want to be taught by a teacher and not my peer.” But in the pandemic moment when they don’t get to connect with their peers, like socially. So our students are back on campus at Wake Forest. But there’s lots of restrictions on what they’re allowed to do with each other socially. So especially for some of our first-year students who hadn’t made friends yet, this was their opportunity. Classroom collaboration was their opportunity to make friends. And so yes, it was tied to their learning. But they also really just appreciated it and said, “I want to be able to work more, they helped me understand the material more.” So they were calling out both the sort of friendship aspect, the social aspect, and saying, “Oh, it helped me feel more confident in the material, because I could ask questions.” So I certainly think it’s not a universal story, that students are upset about these kinds of active learning and small stakes things. But it’s more universal. I think that they feel like there’s a lot more work. And so that’s what’s so interesting. Rarely do you have a finding or experience where so many people are in an agreement about this. And so it’s just such an interesting thing that I have not met a person who said, “I felt like I had less work.” That’s kind of interesting. But there was one student in our survey, I think I quoted this in our blog post, that was really interesting, where she said, “The courses are easier, but they’re emotionally more difficult.” So the online courses are easier, but it’s emotionally more difficult and more difficult to try hard for. One of my hypotheses was that being in a pandemic makes our capacity to work lower. And so I think that’s part of what that person was getting at. Everything feels like more work, even if it’s the same amount of work. And I am guessing that it’s both that and also maybe a little bit more work too, that’s going on, I’m going to be curious to see what happens in the spring. We’re gonna do our survey again. And we did have some interventions where we talked about this, but there’s no mandates about what people are going to do. So we’re going to ask our faculty, again, what they’re doing. And then we’re going to ask our students and see if things got better. And hopefully, that’ll make us understand maybe which hypotheses are more or less likely to be true? Who knows?

Rebecca: If anything, at least, this is something faculty and students all have in common. We all feel like we have more work.

Betsy: Yeah, well, [LAUGHTER] and actually, we didn’t even mention this. And I didn’t mention in my blog post, because it was already too long, is some of this switch to low-stakes assignments also increases the workload for faculty. You don’t have to assess it all, but many of us are just used to that, so we look at everything and grade everything. And so certainly, we heard a lot on our faculty survey of “I cannot sustain this for another semester.” So, this semester, we may find that many of them have shifted back to fewer, larger assignments. So I’m not sure. We just heard some anecdotes, but I could see that happening too, for their own workload sake as well.

John: In addition to the trauma of the pandemic and all the issues associated with that, I believe you also mentioned the fact that many students signed up for face-to-face classes and just being in an online environment is going to make them less happy. And if you’re not as happy in that environment, it’s going to seem like more work.

Betsy: That’s right. Yeah. And this is where I had a throwback to my own time tracking that I did. Maybe five or six years ago, I did time tracking of my own time and I was fascinated because I wasn’t very good at predicting what I was spending my time on. If I didn’t like being in a meeting, I felt like it dragged on and on and on. If I was reading a book that was really exciting. I thought it was like this [finger snap]. But actually, if I went back, “Oh, I was actually spending a lot of time” or even just working on a design project, I would just lose hours staying up till however many hours in the night because it’s exciting to me, it doesn’t feel like work. And so my guess is that there’s some of that going on, too. And I will say in our survey, there was a group of students who were really unhappy with online learning in general, not specific teachers, not specific strategies&hellip that they did not want online. And so those students, obviously, if they had that much anger and sadness about being online, I can’t imagine that they would be excited and enjoying&hellip like, just another 15 minutes of online would be a slog for them, you know, and so I’m sure that things are slower, because they’re not enjoying it, because they didn’t choose it. And I think that’s a really important thing for all of us in higher ed to be thinking about is that, just because there are some students who are unhappy with online right now doesn’t mean that online itself is the problem. It’s partially giving our students autonomy and choice of how they experience their courses. And there are some of our students who just really want to be in person. And those students are probably also the people who really want to be socializing with their friends. And they aren’t getting any of that right now. And so they’re doubly upset, triply upset, like many of us, and that’s not a good position to be in to enjoy your work&hellip the work is really work. I’m sure that some of that’s going on.

John: I spent a decade working on our faculty assembly one semester. [LAUGHTER]

Betsy: I like that. Yeah, there you go. That’s true, right. Sometimes there’s something that just drags on. Time is tricky like that. Some of our students also commented on just sitting in their dorm rooms all day on their computer screen all day, and leaving to get takeout food and coming back. And they’re in singles, often&hellip a lot of them are in singles, because we de-densified our dorms, like it’s just not a great mode of existence. And so anything they can do to get away from the screen, as Rebecca, as you said, that I think is a really valuable strategy for all of us to try to incorporate into our courses.

Rebecca: I’ve noticed this semester, in my classes, I have really good engagement. They’re synchronous online, I can see people contributing. But there’s a lot less camera use this semester than there was even last semester with some of the same students. And maybe it’s the winter slog, “Oh, the winter won’t end.” But it’s just also just being on screen and feeling almost like you’re in performing mode. I think it’s some of that, too. I’d like to turn my screen off sometimes.

Betsy: I was just gonna say that for those of us that are in committee meetings all the time with our colleagues, like we’re still with screens on all day. And yeah, I absolutely think that there’s just an exhaustion and awareness that there’s another semester of this, we don’t know when it’s going to end. &hellipreally tough, certainly.

Rebecca: So we always wrap up by asking what’s next?

Betsy: Well, if I knew what was next for the fall, I’d be a millionaire right now. Who knows what’s next for the fall? I think that’s the biggest challenge for all of us, as we’re thinking about higher ed, in the near term, at least is what’s going to happen in the fall. But I do think with respect to the topic of this podcast, we often talk about when the pandemic ends. It’s going to be like a trickle, I think. There’s not going to be a sharp ending to it. But whenever we start talking about the future of higher ed in a serious way, I do think there’s going to be a very interesting question about how much do we expect of our students outside of class? And what is an appropriate workload? What is the nature of a credit hour? All of those kinds of questions should be on the table because I know for a fact that many of my faculty, even when they go back to in-person are going to want to keep using the strategies. They’ve read the research that we presented to them this summer, and they see that it’s valuable and that their students are learning and so it’s not as if the workload is going to decrease dramatically, I’m guessing, when we go back to in person, so we may need to have larger conversations about that in higher ed.

Rebecca: Thanks so much for joining us, Betsy. It’s always a pleasure.

Betsy: It was great to be back. I love this podcast. Thanks so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]