143. Pedagogies of Care: Creativity

Is creativity something you value in the work that students produce? In this episode, Natasha Haugnes and Martin Springborg join us to discuss ways to spark, motivate, and support creativity.

Natasha has served in faculty and curriculum development at the Academy of Art University and as an adjunct professor at the California College of the Arts. Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College, Natasha and Martin both contributed to the Pedagogies of Care project and are two co-authors (with Hoag Holmgren) of Meaningful Grading: A Guide for Faculty in the Arts.

Show Notes

  • Haugnes, N., Holmgren, H., & Springborg, M. (2018). Meaningful Grading: A Guide for Faculty in the Arts. West Virginia University Press.
  • Pedagogies of Care
  • Haugnes, N., & Russell, J. L. (2016). Don’t Box Me In: Rubrics for Àrtists and Designers. To Improve the Academy, 35(2), 249-283.
  • Haugnes, N., & Russell, J. L. (2008, 2014) “What do Students Think of Rubrics? Summary of survey results: Student Perceptions of Rubric Effectiveness
  • Sawyer, R. K. (2011). Explaining creativity: The science of human innovation. Oxford university press.
  • Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of personality and social psychology, 22(1), 113.
  • Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of educational research, 71(1), 1-27.
  • Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. WAC Clearinghouse.
  • Nilson,. Linda (2019). Specifications Grading. Tea for Teaching podcast. August 21.
  • Tharp, Twyla (2006). The Creative Habit: Learn it and Use it for Life. Simon & Schuster
  • Transparency in Learning and Teaching (TILT)

Transcript

John: Is creativity something you value in the work that students produce? In this episode, we discuss ways to spark, motivate, and support creativity.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guests today are Natasha Haugnes and Martin Springborg. Natasha has served in faculty and curriculum development at the Academy of Art University and as an adjunct professor at the California College of the Arts. Martin is the Director of Teaching and Learning at Inver Hills Community College and Dakota County Technical College, Natasha and Martin both contributed to the Pedagogies of Care project and are two co-authors (with Hoag Holmgren) of Meaningful Grading: A Guide for Faculty in the Arts. Welcome Natasha and Martin.

Natasha: Good to see you. Yay.

Martin: Thanks for having us.

John: Our teas today are:

Martin: I’m drinking coffee this morning.

Rebecca: Always… Always the rebels.

Natasha: Well, I had my two cups of coffee and now I’m on to Wild Sweet Orange Tea…

Rebecca: That sounds good.

Natasha: … and it’s delicious. Yeah.

Rebecca: I have iced Scottish afternoon tea

Natasha: Afternoon? Huh…

John: And I am drinking Tea Forte Black Currant Tea. We’ve invited you here today to discuss Natasha’s contribution to the Pedagogies of Care project and your joint work on Meaningful Grading in the Arts. Natasha, could you start by telling us a little bit about your contribution to the project?

Natasha: Sure. “Nurturing the ‘aha moment’” is the topic of the video made. And it was based on one of the tips in the meaningful grading book that I co-authored with Martin and Hoag. This video focuses on the “aha moment,” or that moment of insight in the creative process, and how to really nurture students and invite them into that moment. I focused on the “aha moment,” which could also be called the moment of insight in the creative process because it really is associated with kind of joy and happiness and magic. And there are a lot of cultural myths around insight and creativity in general, but especially these magic moments. People think they come out of the blue, that they’re come down from God, that they’re somehow related to some innate ability. And research shows us, and people who are creative practitioners know, that this is not entirely true. So, I just decided to kind of hone in on that moment. In my work at the Academy of Art University, I have worked with a lot of students and a lot of instructors who are often drawn to creative fields because of the joy and they really want to engage in that, the joy of the creative process. But then when the students get to school, and when the new instructors come to teach, they often get really drained. And they find that there’s so much hard work and there’s so much stress in the classrooms, even in things like painting and graphic design and moviemaking classes, students seem to get really rundown, and they don’t connect with those moments of joy. So, this results in frustration. At my own school, we were seeing pretty high dropout rates of students at a certain point and I actually ended up working with at-risk students in my role as the Resource Center Director at the Academy of Art University many years ago, and that taught me a lot about working with students and engaging them in their creative process. A lot of the students I worked with, they were sent to me by an instructor who would say “This student is just not engaging. They’re really sloppy in their work. They’re really lazy. They’re not putting the time in.” And when I talk to those students, I would find patterns that really ultimately meant that they weren’t understanding their creative process. They were doing things like brainstorming a whole bunch of ideas, and then trying to finish one, but then getting distracted and thinking, “Ah, I’m going down the wrong path, I’m going to do this other project, I need to take this other approach…,” and they would go down another path, and then they would abandon that path, and they would take yet another approach and pretty soon it’s time to go to class and the project they’re presenting for critique looks like it was done at the last minute. Again, this is really frustrating for the student and the instructor. And I realized I needed to learn a lot more about the creative process in order to work with these students and help them connect to that joy, help them understand how the hard work connects with the joy, and help the teachers understand how the hard work connects with the joy. I think it’s really imperative that our faculty understand creative process and define it so we can teach it to our students. And this is especially important for students whose livelihood depends on creativity, like a game designer, a graphic designer, even an illustrator can’t just go to work and hope that insight comes, they need to learn how to have some control over that, not only for their own work, but just so that they continue to enjoy what they’re doing.

John: It sounds like part of the problem is that people think that creativity is just something that people either have or don’t have, and they don’t see that it involves a process that includes a lot of work. What types of things can we do to nurture students in making the connection between the work that they do and that aha moment to get them to that point, so that we don’t lose them on the way.

Martin: One thing that I talk about quite frequently with faculty, no matter their discipline, but especially in the creative fields, and one thing that we go back to quite a few times in Meaningful Grading, is rewarding failure and grading process versus grading that final product. If you value the development of a creative process and you value your students diving into the waters that they’re sort of murky, they cannot be afraid to do that. And at the same time, they should also be aware that you’re rewarding that effort and their engagement and what can be kind of a scary process for them, especially if they consider themselves non-artists or unable to do art because they don’t have some innate knowledge of it. So, as you develop grading systems, making sure to work into those grading systems those things that you truly value about that process and about your course.

Natasha: I think it’s really crucial. And something that I try to point out in the videos is breaking it down, scaffolding the process for them, breaking it down into small accomplishable steps and explaining to them: “No, this is not creativity, this is not your whole project. This is what you need to do now. And here’s what you need to do, and you need to put the work in to do it. And then you can move on to the next step.” I think that’s really important, and it’s just really important for the instructors to do that. We often have the overview, we understand the process, we have faith that they’re going to get there, but the students don’t, necessarily, and so that’s kind of what leads to those patterns of procrastination that we see with the students who aren’t doing so well. They put things off, they don’t understand the importance of that early hard work that you really have to just put in in order to get the payoff at the end.

Rebecca: What are some ways that you recommend building in experimentation or risk taking into the grading system? Because those are often things that we value in creative fields, but are harder things or things that we don’t always build into our evaluation systems. We might focus more on the principles of design or something technical, [LAUGHTER] because those are easy to measure.

Natasha: You’re a graphic designer, aren’t you, Rebecca?

Rebecca: I am. [LAUGHTER]

Natasha: I think graphic design is actually a really great example of a place where you can get really bogged down with rules, right? I mean, you can approach graphic design almost as a mathematician and just kind of go “ka-chink, ka-chink, ka-chink, ka-chink, ka-chink” and you can create stuff that follows the rules, but doesn’t really have a lot of creativity to it. And I guess one piece of advice, this goes to a recommendation that I’ve included in the video, but really simplifying criteria. Again, if you can break down the steps and have each step just be assessed on one or two criteria, that allows students to kind of say, “Okay, I’ve met the goal, now I can do what I want. I’ve done what that teacher needs to see, [LAUGHTER] and I’m going to pass, and now I can really play with it.” In some research that I did with a colleague of the Academy of Art University quite a while ago, we did this big study, twice actually, called “Student Perceptions of Rubric Effectiveness.” We found a common pattern in students’ responses, the students that really liked the rubrics said that they liked the rubrics because it told them exactly what they did have to do. And then once they checked off all those boxes, they could just run with it, and that was very freeing to them. We can talk later that a rubric is not always perceived that way, for some students, it kind of acts like a creative constraint. But, I think if we can keep the criteria to a minimum, that can allow students to know what they have to do and then have fun with it.

John: One of the things I noticed in reading through your book, and also in what you were just talking about in terms of giving stories scaffolding, is so much of the advice that you give could apply in pretty much any discipline. While your focus is on the arts, students don’t have the same expertise that we do. And the tasks that they’re facing are much more challenging and require much more processing. And they don’t always come in with that growth mindset. Much of what you’re talking about basically, is how to help students move from this binary view that they’re either good at it or they’re not to recognizing that learning is work, and that they can get better as they develop. And it was nice to see how closely this was aligned to the advice we try to give in so many disciplines.

Natasha: I totally agree, John, and actually I was in a conference at the University of Missouri where they actually viewed this video, and the person who was facilitating the workshop that I was lucky enough to be able to attend from the comfort of my own home office here, she’s a scientist, and she actually put up a map of the scientific method and said, this is the creative process and this is not the exclusive domain of artists and designers by any stretch of the imagination. So, I love having those cross-disciplinary conversations. I actually teach writing and ESL, and so I see some crossover there. I guess I’m just reluctant to offer a lot more advice to teachers of physics and math and economics and things like that, simply because I don’t have as much experience with those instructors. I’ve been exclusively art and design skills for a really long time. Martin, maybe you can speak to that. You have a lot more majors at your colleges.

Martin: Especially in those foundational courses, you’d certainly get students coming in at a variety of levels. So, they have past experiences, or they don’t, and those with past experiences sometimes come in with quite a bit of knowledge or experience in the arts. So, they’ve had a lot of high school experience, for example, that puts them at a different level than the other students in your class that are truly beginners and don’t have any prior experience and consider themselves very much non artists. So, one thing that’s important to do, just getting to the practical here, if you’re in an arts course teaching at that foundational level… or really going back to your comments about this crosses disciplines, no matter what discipline you’re in, if you’re teaching that foundational level course, getting everybody at that same base level at the beginning. Purely speaking from past experience here on this one point, I taught photography for about 20 years. And in my intro courses, I would frequently have students come in that had high school experience, and they had learned something and could demonstrate that thing. But, at the same time, they learned it in a, I’m not going to say the wrong way, but in a bad way. They picked up some poor practices from their previous education in that, and so you have to make an effort to untrain that a little bit and get them to that same process that you want everybody to engage in, at that very beginning level. So, that step and that effort also makes those students who are truly coming in as beginners and don’t have any previous experience realize that “Oh, yeah, this is something that I actually have to learn and that everybody has to learn and these students who come in with previous knowledge, it’s not just some inherent skill or ability that they have in the arts. Another thing that I found really helpful, in sort of leveling the playing field and making it apparent to those truly beginning students, is using my past beginning students who have come into my courses with no experience, using their products as exemplars when I’m talking about how I want somebody to do something. So, if I’m talking about an assignment, I’m using examples from, and I’m pointing out the fact that these students came in from like, say, they’re nursing students or their automotive students, or this student came in with zero knowledge, and this is the thing that they produced, and it’s actually an ideal example of what I want you to produce in this assignment. So, using that, and going back to those examples shows those students who come in as true non-native or true beginners, that that level of achievement is possible.

John: I think that was an interesting point, too, that also shows up in other fields. I know people teaching computer science often will note that it’s much easier to teach people who are true beginners than those who had been self taught or perhaps picked up something in a course, where perhaps not an optimal pathway was given to them. The importance of unlearning things, perhaps, or breaking down the structures that people have and replacing them with stronger structures, can be as much of a barrier as people who are struggling just to get to that initial level. And that I imagine is particularly true in the arts.

Martin: Yeah. And going back to what I mentioned earlier about valuing process, maybe they do produce a product, that’s roughly the same result, like if they come up with the same result, but the process that they engaged in to get to there is so much more complicated and convoluted than what you’re trying to get everybody to engage in. So, they do need to go back and learn process. They do need to be at that same level as everybody else in your course.

John: One of the issues that often comes up in discussing creative fields is the importance of intrinsic motivation. Could you elaborate a bit on how we can help develop intrinsic motivation for students in these fields?

Martin: So, another thing that we talk about or that we bring up in Meaningful Grading frequently is the building of a community in an arts classroom and how important that is. That community is the intrinsic motivator. For example, if you make that a primary goal of yours in a course, you would then grade heavy on participating in that community at the beginning, knowing full well that the goal you have is to make that a more intrinsic reward for students and to back off on the grading or drop it all together, that participation component. So, that they not only learned that after they leave your course and after they leave an arts program that an arts community is vital. Like you can’t develop work in some sort of vacuum. As an artist, you have to be engaging with others, but also within your course, it’s just showing them and it’s creating that intrinsic value. Like, what’s bringing me back to this class day after day is not the grades that I’m getting from my instructor, but the vast resource that I have in these 30 other classmates that are able to give me feedback and support. And that also show me what they’re working on… that give and take. So, that’s one example of building in that intrinsic value.

Natasha: Correct me if I’m wrong, Martin, but a huge part of that community is critique. It’s critique discussions, right?

Martin: Exactly, hours and hours of it.

NATASHSS: …and helping students to understand that just getting that conversation, it doesn’t even have to be feedback, but a conversation, and engaging people to talk about your work does build intrinsic motivation. That’s the big payoff that we’re working towards.

Martin: And if you don’t have that tight community in that class, when you get to the middle or the end of that class, when you really want students to be engaging honestly in critique, it’s going to be like pulling teeth. You have to foster that community so that students feel comfortable, that they can open up, they can give opinions about other’s work, and accept opinions about their own work.

Natasha: I kind of want to get into a little bit of that intrinsic/ extrinsic motivation research. And I guess one of the things that got me into this field, and my obsession with grading and creativity, which people kind of look at me and they say “You talk about grading in art school, shame on you.” But the thing that was so confusing for so many of these at-risk students that I worked with before was they were engaging in those conversations, or they thought they were, with their instructor and their instructor would say things like, “Yeah, you know, you’re doing great, keep going.” And that can mean “Keep going. You got to keep working. ] 3 handclaps] But you’re not there yet.” But the student was hearing it as “Yeah, I’ve done it. Good enough.” Right? And so that student would say, “I got a D+, I don’t understand. Like, what’s going on? The teacher likes me…” or “the teacher said I was doing great.” And so they weren’t able to suss out the actual evaluation in those conversations, especially these new students. So, this is where it is so important to actually have grading systems that align with those conversations and that reflect those conversations. Keith Sawyer, he is like the creativity guru who I follow. He’s amazing. He wrote this book called Explaining Creativity. And there are a couple of pages in this book, Explaining Creativity, where he does essentially a synopsis of all the research on the effects of reward and grading on creativity. And there’s some things that we can look at here that are kind of important… that yes, we can extinguish intrinsic motivation with grades, we can do it by giving As for everyone. We can do it by just throwing grades that are completely unconnected to the actual conversations we’re having in class. And we can do it when we grade students and use a whole lot of really judgmental language and convey that judgment. That will all really decrease intrinsic motivation and creativity. But a lot of that early research on intrinsic-extrinsic motivation goes back to the Edward Deci studies, I believe, and he actually did more work on this later. And there’s a more nuanced conclusion that he came to later that when grades and rewards are perceived as information, when these grades and rewards are based on the quality of work that students are turning in, that can actually enhance creativity, and it can really build intrinsic motivation. But even when you’re using grades well, they shouldn’t be emphasized too much. This is the conversation that I often had in faculty development when I was working with new teachers. Oh, come to class, you’ll get five points. Five Points, that’s not why you come to class. You should not be coming to class to get the five points; you should be coming to class because the conversations are important. That’s why we want you here… and just changing the script in how we talk about grades. You need to have a grading system that has a lot of integrity. But, we should not be banging that over our students’ heads all the time, it should be kind of in the background just running along in the background. And what we communicate to students is the intrinsic rewards of all the work that we’re having them do.

Martin: And that’s why your grading system has to transform a little bit over the course of a semester, going back to that grading heavily on participation at the beginning of the course, where you have to get the students to the course to participate in the beginning for them to realize that there’s value in those conversations. If nobody shows up, they aren’t going to have conversations, but then that can change and it can evolve over the 16 weeks or 10 weeks or whatever length your course is.

Natasha: Yeah, and there are those students who really do care about grades I find in art and design school, there’s a certain subset of students who really don’t care, and that’s fine. And so they’re kind of on their own path, and they’re often doing well. But there are those students who really care and there are the students who are on the verge of failing out of school so they have to care. And I find that just understanding that, instructors need to leverage that knowledge to convince students to do stuff that we want them to do… that we know will do them good anyway, right? So if I say, “Okay, you’re going to be really a grade grubber… you want an A do these things,” and they’re the things that they need to do anyway. It’s a way of kind of tricking them into doing what we want. If you’re grading what’s important in your course, it’s going to work out, it’s going to work out for the students who really care about the grades, for the ones who don’t as much, hopefully, they’ll just be intrinsically motivated to understand why they need to engage. But grading what’s important is really crucial in that, I think.

Rebecca: One of the things we’ve talked a bit about is scaffolding and helping provide structures. So if we were to provide structure for faculty who are thinking about the idea of building a grading system that has the values that we’re talking about, things that really they care about or are important to their class, what are some of the steps you would recommend they go through to actually develop that system so it actually does reflect the values that they want?

Natasha: Well, [LAUGHTER] my answer to that is it’s ultimately working towards a rubric. And again, that can be kind of a bad word. I’m the one who’s been walking around the art and design school for 25 years saying, “Let’s build rubrics. And let’s do normings.” And I had a photography teacher tell me one time “Natasha, you gotta understand when you say “norming” to an artist, I mean, that’s like death, you know?” So I’m like, “Ah, sorry.”

Martin: There’s a reason why we don’t have assessment in the title of our book.

Natasha: Yes.

Martin: That was on purpose.

Natasha: It was by design. Absolutely. For the really grade-averse instructors, I start with a conversation. And I usually start with grading because that’s a really good entryway. And I’ll just say, “What are you teaching? And what does that look like? And what does it look like when a student does it? And what does it look like when a student doesn’t do it?” And really, that’s where you start. And then I think the next step is really getting real student work in front of this instructor or this department or this cohort of instructors who are teaching the same course… different sections of the same course. They need to look at the student work and they need to say, “Well, yeah, that one meets the criteria for this course that doesn’t.” Why? Why not? Having those conversations, that’s like the best investment that I think any department or any instructor can make into really focusing their teaching and to improving assessment is just think about how you’re making what you teach visible. And then what does it look like when it’s acceptable and when it’s not acceptable, when it meets the goals and when it doesn’t meet the goals? And then it just moves on from there. And if what you think is important, the quality of the color print in your poster, or the resolution on your screen of your logo, or whatever the heck you’re talking about, it might be process. So again, what does that look like? Well, I want to know that they’re listening to the feedback and really taking it in. Well, how can we make that visible? Maybe I have them do a little recording or do a short paper saying here’s all the feedback I heard, and here’s how I responded. “Joey told me I should change the concept, but I didn’t like that idea because…, so I’m not going with that…” and actually have them make that thought process visible. So, it takes some, again, creativity on the part of the instructor in the field of the teaching and learning. But usually, if there’s something really important that you’re teaching, you can have a way to make it visible and figure out what you’re looking for. And what does it look like when there’s evidence that the student has done what you need them to do? And what does it look like when that evidence is not there yet?

John: I usually meet with new faculty and generally ask them what would they like to see workshops on and, about six or seven years ago, one thing that was requested was a workshop on evaluating creative work. So I reached out and we got four people from different departments. We had someone from art, someone from music, someone from theatre, and someone from English. And they put together a presentation of how they evaluate creative work. And one thing that was in common was they all used rubrics, and they all talked about how there are certain fundamental skills or processes that students have to follow. And that’s what they embed in the rubrics and it surprised a lot of people in STEM fields who were attending because they were much less clear about what they were expecting from students and They expected something that would be much less well defined. And so one of the things they also emphasized, and you’ve talked about is that it’s telling students exactly what they’re expected to do and what types of things they need to demonstrate in the work before they can embellish on that. And that was a really important feature in all of their discussions, the same arguments show up in your book. That surprised many people outside of the creative fields.

Natasha: Oh, those are my tribe. That warms my heart to hear that, John. That’s exciting, yeah.

Martin: One of the added benefits of using rubrics is that time saved as well. Faculty time is a precious commodity. And if you can convince them or just show them how much time will be saved by simply having that rubric available, and using it as a guide, as you’re going through the assignments that are piled on their desk, it’s a convincing argument.

Rebecca: So, we talked a lot about building in values into our evaluation system. Can you talk about some of the things we should avoid doing.

Martin: I can speak to that a little bit. So, one thing that I’ve seen a lot of arts faculty members do… from a student perspective. So, coming up through the arts, one thing I’ve seen a lot of, and heard stories about, is the instructors bringing their personal bias, their own career and background, and that subjectivity in general, to the process of evaluating student work. So I’ve heard some pretty bad horror stories about that. For example, I’ll just go into one story quickly because I think just every faculty member who’s hearing this should know that this is never something that you want to repeat. So all the work, as you can imagine, all the prints, lining the board during critique and the instructor just, without words, just going across the board, pulling work down and throwing it out the window. Like if he doesn’t like it, right… if it doesn’t meet his criteria, which are a mystery, by the way…

NATASHAS: I’ve been in those classrooms. I’ve seen that.

Martin: Tell non-arts people about these stories, and they’re like, “no.” Yeah, it really happened. So remembering that you got to check your personal bias and your personal preference for art at the door and rely a lot on, or more on, having students engage in self evaluation, like did they feel like, and how do they feel like, they have made this, or communicated this, through their work, this issue that they think is important through their work. And if it doesn’t, like if you’re not understanding, then engage in a conversation about it. Like how they feel they’re getting there and where you think they’re not getting there. So using that as a starting point instead of your own, “I am the authority on art, and this is why this does not work.” That’s a huge demotivator.

Rebecca: I think one of those biases that a lot of faculty might bring to the door, is the history of white art created by white individuals.

Martin: This is the history of art, it’s all white male.

Rebecca: If students are creating their work from different cultural perspectives, and the faculty member is not up to speed on other cultural perspectives, we’re enforcing essentially a white supremacist point of view and system. So how do we engage in those moments in a way that’s productive, especially if we don’t understand the cultural background that something is based on?

Martin: Yeah, if students can’t place themselves in the history that you’re talking about, you’re referring to, how are they to imagine themselves in that world in the future?

Natasha: I’m gonna offer just one little tip here because yes, I hear you, Rebecca, and we see it everywhere in the overwhelming influence and sort of self-perpetuation of the white colonialist culture, even in our art classes. Something that we found when we did our rubrics research was that students, in general, really love rubrics, it helps guide their work. But what they really loved… even more than the grid of language… was samples, examples of work, examples of work that span the quality. Here’s an example of something where somebody tried really hard but they didn’t quite hit the mark. Here’s some examples of passing work. Here’s some examples of work that really hits it out of the park. And it’s really important not to have one example, especially in a creative field, because what happens then? The students who are not very competent will copy. Here’s an opportunity to allow for many different interpretations and really show those to your students. Consider using student work from previous semesters from a diverse range of students with diverse content. And that gives students something to connect to, it helps them see themselves in the class, it helps them understand that you, as an instructor, see them and value them. And that even though you have these criteria, there are many ways to reach those goals and reach those marks, those criteria that you’re putting out.

John: And so, by including a range of examples too, from different genres or different approaches, so that it does not become just a Western culture, perhaps. In recent podcasts we’ve done with Kevin Gannon, for example, he talked about decolonizing your syllabus and just suggesting that when you’re putting together your syllabus or searching for examples or exemplars, you could just do a little Google search on decolonize your [insert subject matter here] syllabus, and you can often find some good discussions of that with some good resources that you can build in.

Natasha: Yeah.

Martin: Yeah.

Natasha: This is incredibly important. In my work at California College of the Arts, there’s a very active group of instructors. They’re working on decolonizing the classroom, anti-racism, anti-racist pedagogies, and I’ve learned a lot since I’ve been in teaching there. I haven’t been there for a very long time. But I guess there’s a book called Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future by Asao Inoue. And he speaks quite a lot about assessment. And the point he makes about assessment is he says, in order to really decolonize your classroom, we need to be careful how we talk about quality, because quality so often is really culturally loaded. It’s so loaded that it is really hard for us to even untangle what we see and what we look for. And as a response to that, he really emphasizes grading on labor, grading on the work. And this, again, relates to some of the topics that are in this little video I put together although I don’t really call it this by telling students and taking all that quality judgment away from your rubric and from your assessment and just saying, create 50 of these things, [make 50 taglines, make 50 photographs, write five different thesis statements for your paper or write five different opening lines for your paper and just do that. And that’s the way of just asking for labor. You’re just saying do this work and it doesn’t have to look a certain way or be a certain way. But if you just put some effort into this, you will do well. This is a way of assessing work that actually pans out much better for students of color, students from cultures that are not traditionally represented very well in the faculty at colleges and university. So this is something I’ve been really taking to heart a lot. And in my writing class, I’ve actually, at CCA, where I teach freshmen composition to non-native English speakers, everything is graded on pass not passing yet. And so that really emphasizes the labor. If they’re not passing yet, the implication in that not passing yet grade is that you will do it again. Just do it again. Do it again. Nope, still not quite right, do it again. There have been a few students who have redone their essays four or five times, and it’s painful. But wow, they learn… they learn. And again, the trick is in not having five pages of criteria, but having a pretty narrow band of criteria that we’re looking for here that doesn’t get really niggly about the quality.

John: It sounds like it’s a specification grading system that you’re using. And it’s also building in something much more explicit than the “keep going” message that can be misinterpreted. So giving students the opportunity to try something to not quite get there, but to encourage them to continue working on it more explicitly than perhaps students always hear.

Natasha: And I’m glad you mentioned specifications grading, Linda Nilson has been a huge influence on the way I think about teaching and grading. She’s got a lot of really good thoughts out there for sure.

Rebecca: One of the things that’s really easy to evaluate is something that’s technical that has a right or wrong answer. How do we evaluate in a rubric format, things that are more qualitative, like the amount of experimentation or risk taking or other things that we might value in terms of creativity? Can you give us a concrete example?

Natasha: Actually, we have a a whole tip in our book about risk taking. There’s some really interesting ideas about ways you can really force students into making some mistakes and talking about them. There’s so much that comes up that seems, at first, like it’s going to be really hard to describe it in a rubric. But again, if we just get instructors and people who teach these disciplines together, talking about things, usually they can come up with something much more concrete, even if it’s not a cut and dry technical skill. Concept is one and I have some examples of like before and after for rubric wording. And often when we first write out a rubric, we might use some really sloppy language like “The concept is sloppy. It’s lazy. It just doesn’t work.” That just doesn’t work, right? [LAUGHTER] And so that might be the first draft. But then you start looking at some student work and talk with your colleagues. And you’ll find some more precise language will come out. Often when we talk about concept… I’m talking about the context of maybe an advertising campaign. But the concept is predictable. That’s a concept that is not acceptable is predictable. It’s the first thing that comes to mind when people think of this product. So, that is not a good concept. So there you go. Now we’ve made something a lot more understandable to the students and to the instructors when they’re using this rubric to grade later. And it can help you move forward in a way that that judgmental language won’t. It just makes the students feel bad. It makes the teachers frustrated, because we’re like, “Oh, it just doesn’t work.” But actually taking the time to look again at a range of work that doesn’t meet or that does meet the expectations for this thing that seems really nebulous at first usually you can manage to articulate it, and if you can’t, then maybe that’s not something you’re actually teaching in your class and maybe that’s outside of what you’re assessing. This is another tip that we come up with quite often. I think oftentimes instructors who fear grading, they think that they need to grade the art and you can’t grade art. No, you can’t grade art. You can’t say Picasso was better than Twyla Tharp. You can’t compare people and grade artists in a holistic way. Your grades should be based on what you’re teaching, and the objectives for your class. And we can communicate to our students, this is what we’re looking at here. You’ve also done this other stuff really well, but in our class, we’re really looking at this, so this is what your grade is based on. And that’s a really important factor in this whole endeavor, as well. One other little trap, I think, that faculty members can fall into when we talk about assessing grading or assessing creative work is that when we sit down to write our criteria out often the first thing we want to talk about is that incredible piece that that student two years ago did, it was amazing. It was mind blowing, it was so good and students need to see this and you get into those conversations. And that’s fun to talk about with your colleagues and you pull up that student’s work. And you talk about how great they were and what they’re doing now. Yes, that work should be shared with other students, that’s exciting. We have to celebrate those moments. But for the student in the middle of the pack in your class who’s kind of struggling, we need to think about what’s acceptable. That’s why it’s really important to really focus on that line between what meets expectations and what doesn’t meet expectations, because there are some students that just really need to work on that. [LAUGHTER] There are others that are going to blast through that and do really great things, but the ones that need our help are usually the ones that are hovering around that middle area.

Rebecca: So, we’ve talked a lot about rubrics and grading and evaluation, kind of assuming that we’re living in a perfect little world in some ways. But as we all know, right now, in this moment in time, there’s a lot of extra stress of COVID-19, protests related to Black Lives Matter, and any numerous other health things that are coming up because of COVID-19, remote learning. [LAUGHTER] All of these things, there’s lots going on. And so students are under more stress than normal. Students are often under a lot of stress, but this is like extra stress. So in these moments, what are ways that we can help promote creativity and also help our students really feel supported and being able to learn whether they’re on this point in the spectrum or they’re finding being creative really therapeutic and helpful, and all the way to students who just feel like they’re frozen because there’s so many things going on in the world, they feel like they can’t move forward.

Martin: I think now is a great time to be engaging students in creative process. It’s what gets us unfrozen. I’m speaking purely from my location at a Community and Technical College. If we can get students to engage in those often elective courses outside of their major or area of focus that allow them the opportunity to dive into those things that they are feeling a lot of stress about or anxiety about. It helps students be more successful in those courses that they do have to get through as a matter of course for their program of study.

Natasha: Oh, boy, these are hard times. I think, just most immediately from the video, the nurturing the aha moment, I think that it’s even more important than ever to break down our projects into small steps and help make those steps really kind of distinct from each other. I think that’s something that’s happening for students now, and for us, is we’re sitting and we’re staring at the screen all day long and it can become this big blob of existence where one thing bleeds into the other. And if we can really make the steps a little bit distinct, including a few steps where the students just disengage from all social media and anything online where they can actually be alone, without all of the electronic stimulation. I think those are things that can really help nurture their creativity. And also just I think there’s this funny paradox right now that we’re all alone. We’re all isolated. And yet, if you’re sitting there on your TikTok and Instagram and all day long you’re connected and that can be really, really stressful… and so convincing students to take a break from that, telling them we’re going through another step now. [LAUGHTER] And keeping things again really simple so that they can have that opportunity to use what we’re doing in our classes as a springboard to express themselves. Encourage them to incorporate what’s going on in their own life into the work that we’re doing, including examples and acknowledgments of what’s going on in the world. Really important. And it’s a fine line. I’ve just talked about this with my co teacher about how we’re going to be discussing Black Lives Matter, the latest George Floyd protests, and the Black Lives Matter protests, and the defund the police protests with our students who are mostly from Mainland China. Where do we even begin with that discussion? How do we do that without completely stressing them out, but also using it as an opportunity to feed their curiosity and acknowledge their own stress around these issues? So we need to let them know that we’re a safe space for everybody to engage and really help them break down things into small packages and celebrate their achievements. And again, let them keep working if they’re not quite there yet. Let them do it again. Let them do it again, let them do it again, I found myself being very forgiving on deadlines,

Martin: We also have to help faculty realize that they’re safe to engage in those redesigns and those conversations, and that comes from at that administrative level, engaging this at a college or institutional level. So that you aren’t leaving faculty to figure this out on their own. At my two colleges, for example, we have this new initiative that will run all the way through next year, and actually, for the next three years, probably called Equity by Design. And so we’re starting with a team comprised of administrators, directors, faculty, helping each other understand what this effort is going to be at a college level.

Rebecca: One of the things that you’ve both emphasize is kind of these small steps. And I think a small step for an expert might be different than a small step for a beginner. [LAUGHTER] Can we just take a minute or two to describe the differences between what an expert might think of as a small step and what might be in practice an actual a small step for a student.

Martin: One thing that we have been engaging in at my colleges is the TILT framework of Transparency in Learning and Teaching by Mary-Ann Winkelmes and her team. Mary-Ann came to one of our colleges in January and actually spoke and I’ve been facilitating communities of practice at both colleges on this topic this year. And in that work, there’s a realization as faculty review each other’s assignments and each other’s syllabi that you’re not starting at square one, you’re actually starting at square five, because we have to so often take a step outside of our disciplines to realize that, like you just said… So, what’s complex or complicated to one student is not for another and vice versa. So that transparency effort helps us to really outline the steps of an assignment, even those small steps. And so I’d encourage any family member struggling with whether or not to start at this point or that to review that transparency literature a little bit to engage with their colleagues, share assignments, and ask their colleagues whether or not they’re starting in the right place.

Natasha: That’s such a good question, Rebecca. The expert/novice thing is just something we grapple with all the time as instructors, especially if we’re teaching a new course… something that I’ve had to do in my own class… I was just thrown into a very new course for me a couple of years ago. And we did a new project on public service announcements this last semester, and I start something in class, I told the students “Now, choose a topic from this list of public service announcements that you’re going to create. And first thing you have to do is do some research. So let’s look at some websites.” And by having them do that in class and seeing what they come up with, I start to say, “Oh, right. [LAUGHTER]] They’re going to TikTok, you know, they’re going to these kind of places I didn’t even anticipate, and that allows me to then say, “Okay, I need to actually really scaffold this down.” I don’t want this to take two weeks of my time, I want them to find a credible source and then I ended up giving them a list of basically five places they should look. And you might say that is oversimplifying it, but again, this was just a step in the process of a larger PSA that they needed to make. So I needed to really like clamp that down. But I think if we can have students start in class and actually watch what they do, that gives us a lot of information about how big a step they’re willing to take on. And again, the little creative process chart that I put in the video that I created, I think a lot of creative practitioners, people who are really established, they’ve internalized this process, and they even don’t even want to put it on the line. They’re just like, “Oh, you bounce around, you know, you go back and forth and it’s not a linear thing.” And that’s not actually helpful to a new student who’s really nervous, who’s really stressed, who’s in school for the first time. They’re paying a ton of money to go to art school and their grandparents are really pissed because they should be an accountant. That’s intense. And so these students really need things broken down. And I think that just an awareness of our own expertise is a good starting point, and taking our cues from the students.

Rebecca: This has been really interesting. We always wrap up by asking what’s next? \

Natasha: What’s next, Martin? [LAUGHTER]

Martin: What’s next for me is to finish this book I’m working on with Cassandra Horii. We’ve been doing this project together for the past decade or so. I’ve been making photographs at colleges and universities across the country. We use those photographs that I make in classrooms in faculty teaching to help faculty think about their teaching practice. So we do this form of photo0based teaching consultation. So we’re putting those thousands of thousands of photographs together into a book. And we’re working with the same press that Natasha and I were with, West Virginia University Press, on that book. As far as my other life as an administrator in higher education, what’s next is figuring out what fall semester looks like. How are we engaging students? And in what space are we engaging them? Are courses going to be offered HyFlex, we don’t know? Are any courses going to be conducted face to face? Some of them have to be. You can’t teach arc welding at a distance. There’s some of that that has to be hands on. So figuring out exactly how we’re engaging students in this next phase is what’s next for me.

Natasha: I’m going really micro because these are really big questions. I’m going to keep working on the curriculum for my ESL class. I am now not in faculty development officially anymore at my university in an official role. My current role is that I coordinate and write the curriculum for one level of the English for non-native speakers at the Academy of Art University. And it’s exciting. So I’m working on actually integrating more of the anti-racist ecologies. I’m working on incorporating even more creative process readings and practices into my ESL course in the new zoom world, also really trying to figure out how to get students conversation practice in zoom. That’s the really tough one. So, I’m very much just kind of looking [LAUGHTER] about two feet in front of myself right now. And boy, as far as the bigger issues go, I don’t know. Let’s check in again in the fall. [LAUGHTER]

Rebecca: I think that’s fair.

Natasha: This afternoon, I’m going to make a creative genealogy for myself. I’m making a creative family tree, because I’m having my students do this next week when we start class and I’m going to do it for myself as a sample for them and also just to see what it’s like to go through that process. So that’s actually been really fun. That’s my fun thing that I’m doing.

Rebecca: It’s all about balance.

Natasha: Yeah.

John: Well, thank you. This has been fascinating. I really enjoyed reading through your book, and I’ve enjoyed your contribution to the Pedagogies of Care, and it’s been really great talking to you. Thank you.

Rebecca: Yeah. Thank you so much.

Natasha: John and Rebecca, it’s been a really fun conversation. Thanks so much for inviting us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

140. Pedagogies of Care: Nerd Edition

This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, Dr. Jessamyn Neuhaus joins us to discuss the myth of the super teacher and the importance of focusing on self-efficacy, being human, and being reasonable with ourselves and each other. Jessamyn is the Interim Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is a recipient of the State University of New York Chancellor’s Award for Teaching Excellence. She is also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts and Nerds Who Want to be Effective Teachers.

Show Notes

Transcript

John: This week we continue a series of interviews with participants in the Pedagogies of Care project. In this episode, we discuss the myth of the super teacher and the importance of focusing on self-efficacy, being human, and being reasonable with ourselves and each other.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist&hellip

John: &hellipand Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Jessamyn Neuhaus. Jessamyn is the Interim Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at Plattsburgh. She specializes in the study of pop culture, gender studies, and teaching and learning. Jessamyn is a recipient of the State University of New York Chancellor’s Award for Teaching Excellence. She is also the author of Geeky Pedagogy: A Guide for Intellectuals, Introverts and Nerds Who Want to be Effective Teachers, which she talked about on one of our earlier podcasts. Welcome back, Jessamyn.

Jessamyn: Thank you for having me. I’m really happy to be here.

Rebecca: Today’s teas are:

Jessamyn: I am drinking anything and everything with caffeine all day long, every day since March. [LAUGHTER]

Rebecca: Fair.

John: And I am drinking a ginger peach green tea which is, I think, my fifth or sixth cup of tea today.

Rebecca: I’ve got the Irish breakfast going today. You notice, my caffeine choices are definitely on the higher end lately, too. [LAUGHTER] The powerhouses of tea.

John: Caffeine has been extremely helpful in the last couple of months. [LAUGHTER]

Rebecca: What’s in your teaching tool belt? Some caffeine. [LAUGHTER]

Jessamyn: Yeah.

John: We’ve invited you here today to talk about your contribution to the Pedagogies of Care Project, which we’ve talked about in the last couple of podcasts. Could you tell us a little bit about your contribution to this project?

Jessamyn: Sure. It’s called Pedagogy Nerds Assemble: Battling Big Teaching Myths during Troubled Times. And it’s really about encouraging faculty teaching self-efficacy in the face of so much uncertainty, and trauma, and loss, and struggle now and in the foreseeable future. It also takes the kind of little bit of the snarky tone that I enjoy adding to the scholarship of teaching and learning, kind of real talk to empower faculty to not buy into certain myths that can really interfere with our ability to appreciate our unique ability, our unique contributions to student learning and student success. And mine is in the form of a recorded PowerPoint presentation. I know the project has taken different kinds of format to try to be as accessible as possible. So, I’m very comfortable with PowerPoint, I definitely do not like recording a video of myself. I just did the first one yesterday, because I have a feeling I’m gonna need to do it more often in the semester to come and it was just as awful as I imagined it could be. [LAUGHTER] For the PowerPoint, I have a little picture of myself on the slide, but just my voice.

Rebecca: It sounds like something that we really need. Like self efficacy is something that, in a time when we’re really stressed, is something that we all need more, but also it’s hard to feel like you can empower people to feel like they can empower themselves. Do you have any tips that you can share with faculty about things that they can be thinking about?

Jessamyn: Well, I don’t want to give away all the myths, so I can build interest in the project. But, one of the myths of the three that I tackle in the presentation is the super teacher myth. And fighting that super teacher myth, the impossible ideals of the incredibly charismatic professor who magically helps students learn just by being entertaining. That myth is really, really persistent. And I think the more we can encourage people to recognize that that exists, even maybe at an unconscious level, but to really call it out and recognize it. And that goes a long way towards seeing: “Oh, so here are the ways I can help students keep learning even in these traumatic and troubled times.” I had a crisis pretty early in the shift to emergency remote instruction because I had not taught online before. And I was really struggling with being present to students and communicating to students because, as an introvert who had retreated to her house to replenish her teaching energy, I suddenly found myself needing to open up communication to students at home, while my beloved family (who I wanted to throttle) was humming and buzzing around me. And I had to be more accessible and communicate and present to students, all things I’d learned how to do in person pretty well as part of my teaching persona and to be effective, but I didn’t know how to do it online. And I was lamenting on Twitter: “I suck at this. I’m never going to be good,” and Flower Darby, a scholar of online teaching and learning, reminded me “It took you a long time, like it does for everybody, to learn how to teach effectively in person. The same is true for this new format, this new platform.” And it’s that super teacher myth, “I should be able to do it suddenly, even though I’d never done it before.” So, fighting the super teacher myth would be one of my top pieces of advice. I think.

Rebecca: I think one of the things that you’re pointing out that’s important to remind everybody, as we’re planning for fall in new platforms is there’s a learning curve to anything that’s new. And so if you’re having to learn the LMS, or a new piece of technology, or whatever, the faculty member needs to do that, but so do our students. So, [LAUGHTER] we need to build in some of the time and space to allow ourselves to do that as well as our students and they know when we’re not comfortable or we haven’t built up those skill sets to so being real with students about where we’re comfortable and when we’re not is also not a bad thing being human is important.

Jessamyn: No, and actually it can model for students having a growth mindset, and that learning takes time and it requires making mistakes and, as hard as it is, as difficult as learning is, especially in crisis conditions, especially in the context of trauma and loss, learning is also why we academic nerds and scholarly geeks got here in the first place. I know it’s helped me a lot this semester, in the midst of the struggle and this pain, to be able to look for things that I’m learning about teaching, and I’m learning about my students, and maintaining a growth mindset about my own pedagogical practices, remembering that it always takes a lot of practice, takes experience, takes reflection, but feeling like I was able to learn something… that always makes things better, that makes my nerdy heart happier. [LAUGHTER]

John: I think a lot of faculty have experienced learning in ways that many people hadn’t learned since grad school in terms of making an adjustment. Some people found it easy. The people who are ready teaching online generally found the transition at least smoother than it was for other people. But, for people who were used to only teaching in the classroom, this was a pretty traumatic experience, as it was for many of the students. I was just looking through some comments I got from my students this semester. And some of them said, “I didn’t sign up for an online class, because I really didn’t like it,” and they said the same thing in class right before we made that transition. So, it’s been a learning experience for all of us and maintaining that growth mindset, I think, is really helpful. How can we help students do that? I know you talked about that in your book as well as in your project.

Jessamyn: Well, kind of what I was just saying before, one thing I’ve found helpful is really the modeling portion, especially with the online aspect. And it was helpful with my students, first of all, to clarify this semester. This is not an online class. This is emergency remote instruction, and we’re looking to finish the semester the only ways we can in this crisis condition. And then, just liike we were mentioning before, I also was very clear and upfront about things I was learning how to do. And I’ve mentioned it a couple places now, so it’s getting a little less embarrassing, but I’ll admit it’s still embarrassing. One thing that I was forced to learn how to do was have students submit assignments electronically. I was still making them print out a hard copy of their paper and turn it in, even though for years, I knew I should not be doing that. I knew it made more sense to have them submitted online because I like to scaffold it. So, I always said, so I have to see my previous comments. It totally made sense. Plus, they didn’t have to pay for the printout, which was a real hardship for some of my students. So, I was finally forced to learn how to do it. And I told students, because I made a big deal at the beginning of the semester, I know this is old school and I am being an old Gen X lady here, but can you please print out your assignments? I’m really sorry about the extra step. And then halfway through I said, “Okay, Well, we’re all gonna do this together, and I’m gonna learn how to use the Moodle Dropbox” and I messed it up several times, the settings were wrong and students couldn’t submit. And they were so understanding. A couple of students said this to me, “I know you’re just learning how to do this.” So, it’s okay and it was kind of like modeling that and being clear about this was the technology that was new to me, and trying to be flexible with it. It kind of forced me to also rethink things like I have this really harsh and firm deadline. Well, yeah, except you messed up the Moodle dropbox parameters, so you can’t do that anymore.

Rebecca: I think one of the things that your story illustrates, to some extent, is that breaking down that one myth that you had just talked about, the super teacher, that by showing that we’re learning and that we make mistakes, it also shows students that the learning process includes making mistakes and so it’s not a terrible thing to have that occur. I know that when I’ve struggled with things in class before the students really respond to knowing like, “Oh, I don’t know the answer to that, like, let’s see if we can figure it out.” And the more you can indicate that you’re not some encyclopedia, [LAUGHTER] the more helpful it is.

Jessamyn: Yeah, I had a lot of students clearly very hesitant and fearful about going online. So, helping to demystify that a little bit, I think was helpful.

Rebecca: What are some of your takeaways from this semester moving into the fall?

Jessamyn: For me personally, definitely, kind of building on what I was just saying, being able to better utilize some of our learning management systems to achieve the pedagogical goals that I’ve always wanted to do. There are some very effective tools that I just had not utilized much before because I was doing face to face. One example I can think of is I live for discussion. That’s my favorite part of class and having students discuss, I’ve tried to keep my own piehole shut as much as possible and there are ways to structure, at least some discussion. Even if you’re doing a face-to-face class, you can also include some discussion in your learning management system that’s more inclusive, that will encourage what I hear from faculty lived experiences. And what I’m starting to read about is that there’s ways a good online discussion can increase student participation from people who, for whatever reason, are hesitant to participate in face-to-face discussions. Somebody I know who works with students with English as a second language said when they were forced to switch to online discussions, they started to hear so much more from students who had been hesitant about participating in face-topface discussions. So, my personal takeaway is definitely, when it helps me achieve the pedagogical goal that I would have in any format, I should be able to use some of the online learning tools that are out there. For faculty at large, I guess, I would say two things: One, I saw a lot of pain and struggle, as people were forced to give up things that had worked really effectively for them in the classroom. There’s a real loss there. That’s just one of many, many losses that faculty themselves were experiencing, and of course, in our personal lives during this crisis, but also as teachers, the switch was pretty traumatic in many ways. So, that kind of emotional component and being aware of what we lost and ways that the uncertainties that we’re facing really are going to take a toll day to day, class to class. And the other big takeaway, I think, I saw a lot of faculty and read about a lot of faculty really reflecting for the first time: What are we grading? How are we going to assess student learning? That really rose to the top among the faculty here. How can we possibly assess student learning? They’re just gonna cheat if they’re at home with their book and having it shoved in your face. Well, what do you want them to learn? What are they trying to learn? And how are you going to be able to assess that? So, really deep and difficult reflections on assessing student learning,

John: That type of reflection can result in improved practice, no matter how we’re teaching in the future, I think.

Jessamyn: Yeah, for sure. I just want to give one little shout out here to that term “Pedagogies of Care,” because I do think there’s some misunderstanding about it or assumptions that it means just being completely and utterly touchy feely and a lessening of academic rigor. And that’s not the case, as I talk about in Geeky Pedagogy and have talked about a lot in my own personal experience. You can express care for student learning and a wide range of ways you don’t have to be the extroverted, extra warm, motherly, fatherly professor. I am not that person. I’m very intellectual. And with students, I keep it really professional. But, I’m always getting feedback that she cares a lot, Professor Neuhaus cares so much, because I’m totally fascinated with their ideas and their learning and I do everything I can to help them learn. So, Pedagogy of Care, first for students, means clearly conveying to students that you want them to succeed. And that can take all kinds of different forms. The other thing in my contribution to the project that I emphasize about the Pedagogy of Care is that extends to faculty as well. And we really could all stand to be a little bit kinder and gentler to ourselves and to each other in these extraordinarily difficult times. The Pedagogy of Care extends to our own learning, and not “I flunked. I flunked the semester of teaching. I suck.” No, be as kind to yourself. I’ve repeated this to a number of people for the past four months, like just talk to yourself the way you would to a struggling student that you want to succeed, you know, you’re trying&hellip keep going&hellip you can do it. Don’t give up. This is an obstacle and it’s hard, but you’re learning. talk that way to yourselves too, and try to extend it to colleagues if you can.

Rebecca: I think that one thing that I heard a lot of faculty talking about in relationship to this idea is what they need for self care, and what they actually need and be able to kind of articulate it on a day-to-day basis beyond just the crisis, but there’s competing interests of like family and work and home space and workspace and what have you. And I think people are realizing what kind of time they need for different things to feel balanced, because everything got so out of balance, [LAUGHTER] going from one extreme to another.

Jessamyn: Oh my gosh, yes. I wrote about that. I had an essay in the Chronicle of Higher Education, and it was talking about being an introvert working from home and contrary to this kind of knee jerk: “Oh, introverts have it so great now, because they get to be at home.” Well, except that there could be other people there as well. [LAUGHTER] And demanding, finding, wresting out some solitude when you’re working from home was to me really vital, and it was not easy at all… it was stressful.

Rebecca: Yes, I remember reading that article and thinking, “Yes, all of these things.” [LAUGHTER] We have a system at my house now and that system is really helpful.

Jessamyn: That’s good. Yeah. structure. Yeah. And I live with two off-the-charts extroverts, like off the charts. And normally that works pretty well for us as a family. But, during this situation&hellip no&hellip social isolating. Like our needs were diametrically opposed. I need more time alone. I need more human contact. Yeah, it’s been rough. It’s been rough. [LAUGHTER]

John: We’ll include a link to your article in the show notes as well.

Jessamyn: Thank you.

John: So, you’ve picked a great time to be taking over a teaching center. [LAUGHTER].

Jessamyn: &hellip just the status quo, same old, same old, nothing really big going on.

John: So could you talk a little bit about what you’re planning to help prepare faculty for the uncertainty of the fall semester?

Jessamyn: Sure, It’s a great question. It’s definitely a uniquely challenging time to be trying to revitalize a teaching and learning center on a small rural campus with very limited resources and, like most state schools, facing some really severe financial and student enrollment problems, like maybe forever altering some structures. So, it’s tough. There’s a lot of managing of expectations and emotions. I think the advice that I’ve gotten by far the most, and makes the most sense as well, is the importance of building connections and building communities on campus and reaching out to a wide variety of stakeholders, including students, and really trying to foster pedagogical communities of practice on campus. So not trying to, again with the superhero theme, not trying to fly in and say I’m going to fix everything, but instead trying to encourage sharing of ideas, sharing of resources, support for each other, at this difficult time. We have a very small technology enhanced learning unit that has one instructional designer but we are collaborating on summer programming and working together and just trying to help everyone, I guess, really cultivate that growth mindset we were talking about, and try to approach this as an opportunity for learning. I won’t say silvered lining. That’s not how we want to frame it. But there is this opportunity because every campus has a small group of people who are bought into faculty development from day one, and they’re at every workshop and they want to take every offering and they’re your biggest fans. Then there’s a small group who are going to actively oppose faculty development in any way shape or form and will never ever come to your stupid pointless workshops for any reason, not for love or money. But, then there’s this whole middle population who you could maybe attract them, they could go to one workshop and find it useful and maybe go to another one. Well, that population, in the past three months, has just shot into a whole new world. And I have had, just in the past couple months I’ve been the Teaching Fellow for the CTE and then just starting this gig as the interim director. So I was doing some things with the CTE, and I saw more faces and heard from more people who had never darkened the door of the Center for Teaching Excellence appear and ask me questions and show up. Because, I think it’s not just personally “I don’t know what to do,” but suddenly everywhere, like literally everywhere are professors saying, “I could use some assistance with this. I’m not sure what to do.” Like for the first time in ever, there’s this like cultural acknowledgement that “I don’t know everything.” Like, that’s a major leap for academics to be like, “Well, yeah, maybe I don’t know everything here and I could use some assistance,” but everybody was saying it, everybody was doing it. So, there’s this opportunity to keep building on that and to offer assistance and encourage that growth mindset about their own pedagogical practices to a whole group of people who have never thought about it that way before. So it’s this precious opportunity. I hope I don’t blow it. [LAUGHTER]

John: I think we all share those thoughts about hoping we don’t blow it in getting ready for this.

Rebecca: Now, let’s not put unreasonable expectations upon ourselves.

Jessamyn: I know. [LAUGHTER]

John: But this is true not just for teaching. As human beings, we tend to do things as we’ve always done them unless there’s some compelling need to change. And when things don’t work the way they used to, it forces us to reevaluate how we’re doing anything. And then it’s a great growth opportunity. And it opens a lot of potential. It can be really difficult, as we’ve all been noticing.

Jessamyn: I do think it’s also, not to slam my beloved academic geeks, but I think it can be especially hard for scholarly experts. I mean, we were trained in graduate school: “You don’t reveal your vulnerable underbelly to the alpha academic or you’ll get your throat ripped out.” You always have to be the smartest person in the room. Like that’s the goal of getting your PhD and to back up and admit, “Well wait, I could use some help with this&hellip that’s a big leap for a smarty pants who’s used to their classroom kingdom where nobody ever questions their expertise and authority, which by the way, is not every professor,it depends on your embodied identity. That is a big caveat there. But, you do have this professorial authority and saying “I need help” or saying, “Wow, what worked before isn’t going to work here.” That’s a major leap. That’s a big ask for many academics.

John: And it can help break down that super teacher myth that you mentioned earlier.

Jessamyn: For sure. Yeah.

Rebecca: I think, along those same lines, too, it’s like that same group of people is recognizing all kinds of barriers that students face that weren’t so visible before.

Jessamyn: That’s very, very true.

Rebecca: So really, like transitioning the perception of the ivory tower to something a bit different, and I actually really hope it sticks.

Jessamyn: Yeah, me too. And that’s been amazing, actually, the way I’ve seen that on my campus as well. And it was interesting because I was doing a department-based needs assessment before the emergency pivot. So, I’d been talking to faculty about what they saw as teaching challenges and the student population. And then, within a few weeks after our shift, I saw some of those same professors saying very different things about their students and seeing them in a very different way. Like straight up saying, “My students lives are so hard…” like the obstacles and the lack of access to WiFi, for example, that’s a serious issue. Yeah. And it always has been. So, yeah, it really did. It humanized, I think is the term I hear some people saying is it humanized our teaching in new ways, for sure. And that could be a reach sometimes. Like I was saying, I am very intellectual, I don’t have a lot of personal discussions with my students. But, in these crisis situations, I was very clear about being worried about them and being concerned and hoping they were safe. And all my students appreciated it, but I knew some of them were like, “This is Professor Neuhaus saying, ‘Oh, I’m worried about you. Stay safe.’ Professor Neuhaus, really?” So, yeah, humanizing our interactions is important.

John: I hope that’s a message or a lesson that continues through into the future.

Jessamyn: Me too.

John: And I think it’s worked both ways, that I think a lot of students have seen some of the challenges their professors have faced, not just in terms of using the technology or teaching in a new format, but in terms of having children or pets or other things, or having technology issues, or having access issues themselves, where someone might be using a video game or something similar, cutting into the bandwidth. Many faculty have reported to us that their students have expressed concern, asking if they’re okay, and encouraging them to stay safe and so forth.

Rebecca: I think it’s also along those same lines brought to light some of the invisible barriers that contingent faculty have, being at multiple institutions or the incredible workload that they’ve been asked to bear without really any compensation for the time and effort and energy that’s gone into it.

Jessamyn: Yeah, for sure.

John: Our institution has provided loaner computers and other types of technology to both students and faculty. And an interesting phenomena was that there were more faculty who requested computing equipment and other tools than there were students even. They provided a good deal of it to both, but some of those barriers are not just for students, especially are adjuncts who often are struggling to get by.

Jessamyn: For sure. I was just going to reference and you can put this in the notes to Cate Denial, a historian who wrote a very well known essay, I’m hoping she’ll write a book, advocating a pedagogy of kindness. And I definitely saw how effective that can be this semester for me personally, but I also saw with a lot of other faculty for the first time really seeing what a little bit of flexibility and a little bit of kindness&hellip again, not lessening academic rigor, but bringing in, specifically, some of that humanizing kindness&helliphow effective that can be. And actually, on a similar note, the advocates of ungrading have gotten a big boost as well, because I’ve seen and read a lot of faculty saying, “Wow, you know, once I told my students pass-fail, for example&hellip Wow, their final projects were so great, like they actually did what I want them to do and learn what I want them to learn once the stress and anxiety and kind of false dichotomies, I guess, of grading were taken off the table.” So there’s some real possibilities there.

John: We’ve talked quite a bit about things that we should be focusing on in terms of getting ready for the fall. But what are some things we should probably avoid as either faculty or professional developers in preparing for the fall semester?

Jessamyn: I think a big one is to not ignore or try to just sort of skip over the trauma and the loss that people experienced and also not play like “Who had the worst trauma?” or “Who had the worst loss?” In all kinds of ways we experienced loss as we experienced trauma&hellip and the way trauma works, the weight loss and grieving works is even a small loss can be very difficult because your brain and your heart and your soul are trying to process all your losses and all previous grief and loss. I know I always love graduation. And even though I sit there&hellip it’s very long… it’s very hot… [LAUGHTER] and it can feel like a chore at times. But, the commencement ceremony is so meaningful for everybody, but especially for first-generation students and their families. And we tried to fill in a little bit with some online messages and some online rituals. And I started watching it and just started crying. And I’m like, “What is this? What is happening here?” &hellipand it’s a loss to not be able to engage in that ritual. It’s not the world-ending loss, but it’s a loss and it’s a trauma and people are going to arrive to classes in late August, whatever format it is, with all those things having really just happened to them&hellip faculty, students, administrators, I mean, everybody’s gotten a really raw deal this semester. And that’s not just going to be magically fixed, even if somehow we’re back to exactly where we were. And if all the face-to-face classes are in session as they were planned, and everything’s fine again, but what happened this semester is still gonna be there.

Rebecca: I think that’s a really good reminder. Our students are going to be changed and will be different. And I think in the moment of this semester, a lot of students weren’t able to process what was happening. So, you might really have a really different experience with students in the fall, when they’ve also had a little space to process what that experience was like and the things that they missed out on and are missing out on if they’re online in the fall.

Jessamyn: Yeah, the first chapter in Geeky Pedagogy, advocates for practicing awareness, and really just being as fully mindfully present to the reality of what’s happening around you. And I think that’s always important. But, I think it’s going to be especially important in the fall. And it doesn’t mean we shouldn’t try to change things that are bad, but to first really be cognizant and aware of what is happening, what is going on here. And the state of all our mental, emotional, and physical states is going to be something that we really have to pay attention to.

Rebecca: I think that’ll really shift what first days of classes look like in the fall.

Jessamyn: Yeah, for sure. The uncertainty remains. We don’t know what’s going to happen. And we can put plans in place, but we just don’t know. And that’s…

Rebecca: &hellipterrifying.

Jessamyn: Yeah. [LAUGHTER] For people who love to plan, and I had really fooled myself, you could see it in my book, too. I’ve attained a new enlightened state where I can roll with the changes and you got to be aware of stuff but then as soon as my world was severely disrupted, no, it was all gone. Just zip… gone. [LAUGHTER]

Rebecca: Emotions do have a lot to do with how we learn and process things.

Jessamyn: Yeah! [LAUGHTER] Come on, emotional learning… yeah.

John: As you know, we always end with the question: What’s next? A question that we’re all thinking about pretty much all the time.

Rebecca: Please tell us.

Jessamyn: Yeah. Wow, that’s crazy&hellip ‘cause I have been working on a project, an anthology of insights into effective teaching from women, marginalized, and underrepresented faculty. It hasn’t been suspended, but I extended the deadline, not having the bandwidth and assuming potential contributors also just utterly overwhelmed. And then I guess it’s like day by day&hellip What could maybe help a few people on campus teach effectively? And, of course, how am I going to prepare my own class to be as resilient and flexible as possible for the fall&hellip and just on a personal note, what about my child? He just finished his first year of college. It was not a overwhelming success. I mentioned last time I was here that he is a lackadaisical student. And he had many of the challenges that first-year college students face. And then, of course, this semester has been a disaster. He was one of those students who said, “I don’t want to do an online class.” He’s an extrovert. He’s very social. So, we’re not sure what’s going to happen for him in the fall. So, those are all the uncertainties awaiting us.

John: I first heard about the Pedagogies of Care project when one of the people participating posted a picture of the Zoom screen with all those people in it. [LAUGHTER] And I recognized all of them, and many of them had been guests on our podcast. So, the initial image didn’t talk about what it was for, it just said a gathering of present and future authors from West Virginia University Press, and it looked like a really impressive group of people. So, we’re very much looking forward to this project. We’ll include a link to that in the show notes.

Jessamyn: They’re really some of the smartest people I’ve ever met, and definitely the best collaborators I’ve ever had. It’s a unique experience being part of that series. I’ve never had a group of scholars who’ve kind of come together and really formed a supportive and encouraging community. It’s just amazing. I’ve never experienced anything like it. And this project, I think, is a good example of how the series at West Virginia University Press edited by Jim Lange is unique to not just the scholarship of teaching and learning but to scholarship period, because I’ve been in various series and journals and stuff, but there’s never been a sense like, this is a real pedagogical community of practice where ideas are debated and shared, and each scholar is really supported and I’m really incredibly grateful and proud to be part of it.

John: And that also shows up in the Twitter conversations that take place. For those who don’t follow the authors there, we strongly encourage that.

Jessamyn: Absolutely. Yes.

Rebecca: Definitely, that’s how we found out about this project.

Jessamyn: Thank you so much for having me.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

132. Pandemic Pivoting

The unexpected shift to remote instruction during the spring 2020 semester in response to a global pandemic disrupted established teaching patterns, forcing many faculty to rapidly learn new tools and techniques of engaging their students. In this episode, Dr. Betsy Barre joins us to discuss what we’ve learned from this sudden shift to remote instruction and how we can better prepare for the uncertainties of the fall semester.

Betsy is the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won, with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator.

Show Notes

Transcript

John: The unexpected shift to remote instruction during the spring 2020 semester in response to a global pandemic disrupted established teaching patterns, forcing many faculty to rapidly learn new tools and techniques of engaging their students. In this episode, we discuss what we’ve learned from this sudden shift to remote instruction and how we can better prepare for the uncertainties of the fall semester.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Betsy Barre, the Executive Director of the Center for Advancement of Teaching at Wake Forest University. In 2017 she won, with Justin Esarey, the Professional and Organizational Development Network in Higher Education’s Innovation Award for their Course Workload Estimator. Welcome, Betsy.

Betsy: Thanks, I’m happy to be here.

John: Today’s teas are:

Betsy: I am not having tea, but I am having a raspberry lime Spindrift. I actually would love to have tea, but I just didn’t get downstairs in time, so I have my Spindrift here.

Rebecca: That sounds good. I have an English breakfast.

John: And I have oolong tea today.

Rebecca: Oh, you’re switching it up a little.

Betsy: Sounds exciting.

John: Amazon helps. [LAUGHTER]

Rebecca: Betsy, we invited you here today to talk a little bit about the work that you’ve been doing at Wake Forest to help faculty prepare for pandemic teaching. Can you talk a little bit about what the Center for Advancement of Teaching’s approach has been and what it will look like leading into summer and fall courses?

Betsy: Sure, I can talk more about what we’ve done, sort of what we’re planning for the future is still in process, as I’m sure it is for many institutions. One of the great things about Wake Forest is that our Center for the Advancement of Teaching is not the only office that has been working with faculty and faculty development and digital technology issues, academic technology, etc. So one of the first things that we did when we knew we were transitioning to online, or transitioning to remote teaching, let’s be specific there, is that we pulled together the offices that were adjacent to our office. So we pulled together the Office of Online Education, the Office of Academic Technology, which is an Information Systems RIT wing at Wake Forest, and also we had a number of librarians who did work on digital pedagogy. So we pulled all of us together and created a kind of super team that would support faculty, and that was really helpful to do that really quickly because it expanded our reach, the numbers of folks who could work with faculty and integrated it, so faculty didn’t have to go to a million different places, there was one place that they could go. We had about 850 faculty or so that were teaching that we had to work with and there were about 10 of us on our team. So, it’s a better ratio than some schools, but it’s still a pretty not ideal ratio, and so we tried to streamline things as quickly as possible. So, like many schools, we created a keep teaching website that had resources, but we also created a blog that had daily updates. So every day, they could subscribe to an email and get it in their inbox every morning that would have daily updates, but also resources, tips, things we’d heard from faculty, etc. That turned out to be really helpful and we’re still keeping that going, and it’s been helpful as they’ve been teaching. We also, though, really wanted to encourage them to share their expertise with each other. So, that week that we had off to help our faculty prepare, we did a series of open labs, where we were there to answer questions, but they could also share with each other what they were doing. And then sort of unexpectedly, a few things that we did that have gone really well is that those of us that are on social media saw some faculty talking on Facebook about this, we thought, “Hey, let’s just create a Facebook group,” and that group has been incredibly active. We have like over 300 faculty that are in that group now and some of our professional staff and it’s been a way of communicating. We’ve tried to communicate it outside of Facebook for those who don’t like Facebook, but certainly it’s been a wonderful way of building community that I think will live on after this, and so that has been nice. And then of course, our one-on-one consultations that we’ve always done, but we set up a easier streamlined system for requesting a consultation, and it would cycle through all 10 of us and sync up with our calendars, and so we found that to be really successful, and as successful as we could be in this trying situation. Summer and fall, a much more interesting wrinkle, that we’ve been working on. Once we got faculty up and ready to go, we now could transition to thinking about how are we going to support faculty in the summer and fall. And one of the things we’ve been saying all along, many institutions have, is that what we did in the spring where we had one week to transition is not really robust online teaching. At the same time, we don’t necessarily have the staff and resources to transition all of our courses online for the summer in a robust way, but we have more than a week. So we’re trying to hit some sort of middle sweet spot where it’s not exactly what we would ideally do with online education at Wake Forest, but it’s better and more intentional and takes more time than what we did for remote teaching. So currently, we’re planning for those who have volunteered to teach in the summer to run a three-week course for them to take asynchronously online to learn more about teaching online, and then we’re also gonna offer all 10 of us to do one-on-one consultations and some minimal instructional design work with them. Fall is still up in the air and we’re not really sure what’s gonna happen with the fall, but I think we’ll probably know in the next few weeks what we’re planning.

John: It’s interesting to see how similar the approaches of various institutions have become and a lot of it, I think, is social media made it easy to share some of those thoughts. We also have a Facebook group, we’ve also done lots of meetings and we’ve had a number of people working with us from our campus technology services in providing support and workshops, and it’s been nice to see everyone come together to help so many faculty make this surprise transition that they never expected and didn’t always entirely welcome, but they’ve been really positive in terms of how people have approached it.

Betsy: I agree.

Rebecca: One of the things that I saw you guys doing that I thought was really interesting was “Ask the CAT,” can you talk a little bit about that program and how it works?

Betsy: The name of our center is the Center for the Advancement of Teaching. We still haven’t decided at Wake Forest if we want to do cat or CAT, but we often joke a cat would be funny because then we could have all these funny cat jokes associated with that. But outside of the blog, we started getting some really simple questions that we realized would be helpful for everyone to hear the answer to. And so early on, a few people asked some questions, and we said, “Can we turn this into a sort of Dear Abby letter that we can then publish responses to, really quick responses on our blog?” and they were happy to do that, and then we turned it into a formal themed series in the blog where people can submit online, ask the CAT questions, and they can do it with a pseudonym, so there’s no stupid questions, any sort of challenges they have, and it’s gone pretty well. And we hope to continue to do that because I think we’ve seen on the Facebook pages, I’m sure you all have as well, is that often there are many similar questions, and so when they see us answering another question, faculty get ideas and say, “Oh, I could do that, now that makes sense.”

Rebecca: It seems like the ability to have a little bit of anonymity there in asking the question might allow for some questions that really need to be asked actually be asked.

Betsy: Yeah, sometimes they’ll just ask a simple tech question and we try to expand it a little bit beyond that to say “Okay, that’s great. Here, I’m going to give you your answer, but before I do, let’s talk a little bit about pedagogy and how you might think about universal design,” or something unrelated to the specific tech question.

John: Rebecca mentioned that you had won an award for your work on Rice’s Course Workload Estimator, which is something we recommend to our faculty regularly and people find it really helpful. How would you recommend people interact with that tool during situations like the pandemic, especially for people who are adjusting very rapidly from one mode of instruction to another?

Betsy: Yeah, so one of the things I shared with Rebecca before we started is this actually is really great timing for you to ask about this, because Justin Esarey is the co-author, co-creator with me and I… he’s my husband actually, we did it together… one of the things we’re thinking about doing in the next couple of weeks is actually revising it in a number of ways. We’ve had a long standing interest in doing it, just haven’t had occasion to do that, and there are some changes we’re going to make that aren’t specifically about online, but one of the changes we’re hoping to do is to actually create some categories that are related to traditional online assignments. And again, these are going to be guesstimates. I always tell people, this is an estimator, it’s not perfect. It’s just our best guesses. But to create some estimates of “How long would it take to have a discussion board if they have two posts, 500 words,” sort of things that we’re used to assigning in online education to hopefully help in that regard. But one of the things that I think is a reason this estimator is important is one of the things we’ve seen, and I’m sure you all have seen as well, after about the second week of remote teaching is that some students started to complain about workload, how much work these new remote courses were. And I think part of that is because faculty were incorporating more accountability measures into their courses, so they may have been expecting that work, but never were really holding the students to account to do that work. And so now students actually have to do and show their work, and so whereas before they might have been able to just show up in a lecture, study on their own time, or not study as the case may be, not do the reading as the case may be. Now, if they’re having weekly reading reflections, they actually have to do the reading and that significantly shifts how much work they feel they have to do. So that’s putting it on the students, but it’s certainly the case, and part of the reason we made the estimator, is that as faculty, we’re not really good at estimating how much time our work takes. And that’s true in a traditional setting, it’s true for me, that’s why I created the estimator. I am a humanist, and so I assign a lot of reading and I never really knew, like, how much time it would take them to read, and so that’s what motivated me to investigate the research on that. I think it’s particularly true that we’re not good at estimating how much time things will take when it’s a new assignment or activity that we’ve never assigned, and that’s what we see in this scenario, many faculty are introducing completely new activities and assignments that they’ve never done before. And they often might think, “Oh, yeah, I should give them discussions in a discussion board,” without taking into account how much time that will take, or “Oh, I really want them to make sure that they connect with me each week, in this way,” or “I need to make sure we have these office hours and they need to watch these videos, but since they’re watching the videos, now, we can have some discussion in class because the videos are no longer part of the class time.” And so we think we’re pretty good at sort of keeping track of that, but it turns out one of the things we found with our estimator is that when we asked faculty to play around with it, that we were often very wrong. Faculty were often very wrong about even their own estimates about how much time they thought they were expecting of students. So, I think it can be a valuable check. It’s not perfect, it’s not exact, but it can be a valuable check on our intuitions about how much time we’re expecting of students, particularly with some of these unique activities that we’re asking them to do online, and I also think there’s some really creative strategies by our friends in online education to help us think about a traditional assignment and how to make it a little bit more efficient, discussion board a little bit less time intensive, that we can talk to faculty about as well.

John: With the pandemic, I would think, some of those calculations based on online classes where people intentionally were in online classes might be a somewhat different situation when people are in households where there’s more people in the room perhaps, or where they’re sharing network access, or where there’s more distractions and noise than the people who had intentionally chosen the online environment.

Betsy: Yeah, I think that’s a really thoughtful insight. Absolutely. I think we’ll hopefully get to talk about this later in our conversation today, is that there are a variety of changes that take place here that are not just about the modality, but thinking about our students’ situation, how long it takes to learn the technology if they’ve never learned it as well. Like, “How do I upload this? How do I take an exam?” And so if we give them a certain amount of time for an exam, recognizing that they didn’t choose to do it, they also don’t know the technology as well, and so how do we account for those adjustments as well, for sure.

Rebecca: Yeah, I agree. I think all of those little extra things that now students have to do, including learning the technology or just getting used to a new system or a new rhythm, they all take time. In a semester, we think that’s what the first couple of weeks of the semester are, but then like this semester, we had two sets of those.

Betsy: Absolutely. And I mean, the fact that we are still sending out posts giving suggestions, means that some faculty are still changing things, they’re still adding new things, because they want to try something new or something didn’t work, and normally, we encourage that, but in this scenario, it’s particularly challenging for our students if new things keep getting piled on over five courses or four courses that they’re taking.

Rebecca: If we’re thinking in a traditional context where there’s in class and out of class work, and now everything is remote, how do we think about dividing up that time or what kind of time they should be spending on what kind of activities?

Betsy: I think this is a really good question. And again, my colleagues in online education who think about this question a lot have more subtle distinctions to make about this, but I actually was just having a conversation last week about what accreditors require and how to think about, quote unquote, contact hours in an online environment, and incidentally, one of the things we found actually, unexpectedly with our Course Workload Estimator… again, the motivation was for me as a humanist to basically answer the faculty question of “How much reading should I assign?” was a very narrow purpose, how much reading should I assign? But what we found is that the biggest usage were people who were instructional designers in online programs, who were interested in this question of “How much time is faculty contact hours, is it actually comparable to the face-to-face courses?” So it is connected, and so I’ve been talking about this a lot, and one of the things that, at least the federal guidance suggests, is that one credit hour is about 45 hours of work for students. So over 15 weeks, one credit hour, you do two hours out of class for every hour in class over 15 weeks, and so it’s about 45 hours. They don’t really enforce it, it’s a complicated question or a comparable amount of work, but that’s an easy way of thinking about it. It’s about 45 hours of work for a single credit hour and then 15 of that is expected to be in the presence of the professor. So traditionally, that would mean 15 of that you go to class, 30 of it’s at home. That’s the traditional model that we think of, but in online, of course, it’s different because everything is at home. So one thing, you could just say, “Well, everything’s at home. So then professor never needs to be engaged,” like, you can just say, “I’m going to record all my lectures, put them all up, and then I’ll grade your exam at the end.” Of course, we know that that’s not good pedagogy, online or otherwise. And so I think the way to think about this is, of the 45 hours of work your students are doing, are at least 15 of those hours, somehow engaging with the faculty member? But that could be, for example, a discussion board where the faculty member is in the discussion board engaging and providing feedback. It could be one-on-one sessions where you work on a paper together with the student in an office hour. There are a lot of ways you can imagine faculty presence and engagement that don’t have to be “Let’s have a synchronous video conference session.” But there are some good reasons for that too, particularly in the remote environment where students want some continuity to what they’ve already done. But I think that there should be more flexibility and I think there often is in good online program about what counts as those contact hours, but without just saying, “Oh, as long as we have a video, that counts as a contact hour.”

Rebecca: Along these lines, do you have any advice about designing learning activities and assessments when we have no idea what the modality might be in the fall?

Betsy: That’s a good question. I’m sure that many other people have been asking that question, I myself am teaching this semester, so it has been interesting in helping all the faculty but also teaching myself and figuring out what’s working and what’s not. And I think Derek Bruff at Vanderbilt had a, I actually liked this language that he shared initially about creating pivotable courses. He ended up changing it, he didn’t like that one as much, but I actually like that, like your course could easily pivot. And I think for me, one of the things that I saw was that my course, even though it was a face-to-face course, heavy discussion course seminar course, I had built in already some asynchronous activity outside of class, they were already annotating the text via Hypothesis, which is a really wonderful tool for those of you that don’t know about that in the humanities or any text heavy discipline, Hypothesis is wonderful and in that sense they were already used to and had learned how to annotate their text digitally in the face-to-face course. When we transitioned, it was easy. Okay, we’re going to be doing that. And that was already built into the course. I also think getting all of our courses so far as possible into a digital environment, whether that’s an LMS, or Google or whatever you prefer, can be an easy way too, because a lot of the time we spent with faculty was just getting them to like, “Oh, how do you collect assignments? Okay, let’s get you into the LMS. Here’s how you collect assignments. Here’s a way that you can think about sending a message to students that’s not just through email.” And so at the very least, if we all get in our LMS, or another digital environment, if you don’t like the LMS, and then think of some activities and engagement that our students can engage in at home with each other, or perhaps with you that’s outside of the regularly scheduled class time, you’re already making it easier to shift. But I also think one thing, and we may come to this when we talk about grading, one thing I’ve been thinking a lot about is I had to scrap one of my activities in my course when we transitioned to remote and I’ve been thinking about the particularly challenging situation for those faculty who had a semester-long assignment. So luckily, my assignment was at the second part and so they haven’t started it yet, we can just do something else, because that would be difficult. But if you have many semester-long assignments, that disruption can be really difficult, but if you could organize your course another way to make it pivotable is to organize it in modules, like really intentionally, not just in Canvas, but actually say, “Okay, we’re going to work on this unit as a self-contained assignment that will be done in two weeks. So that way, if we have to take off in week three, you’re already finished with that assignment in that module.” And then there’s one module that’s remote and then if we come back, hey, we get to start another module that might be face-to-face, and so it gives you some flexibility. If you design your course in a more modular way to prepare for disruptions rather than thinking about it as multiple whole semester-long assignments.

Rebecca: It’s interesting that you say that, Betsy, because I’m not teaching this semester, but I’m planning for my fall class, and I teach web design primarily and I was thinking about teaching agile design. So I decided that I would teach it in an agile fashion, which is really what you’re describing. [LAUGHTER]

Betsy: Yeah, that’s smart.

Rebecca: So, I started mapping out what that would look like in these little sprints to work on a larger project, and we would do maybe two projects, one that was collaborative and one that was individual, but in sprints that would rotate between the two projects. So I’ve been mapping out what that might look like, and my real reasoning for that was, specifically if something was going to be disrupted or if it was going to be online, I thought it would be a little easier to help students through the project if it had these clear checkpoints and finishes to things before starting something new.

Betsy: One of the things that made me start thinking this way, and this goes back to the question of how we’re preparing for fall and all the scenarios that all the institutions are thinking about, Beloit college just decided that they were going to actually teach their fall semester in two seven and a half week sessions, essentially. So basically, students will take two courses for the first seven and a half weeks, and then two courses for the second seven and a half weeks. Certainly it’s a lot of work on the part of faculty to transition their 15 week course to a seven and a half week course, but it also is creative because it means that we have to start late, only two classes are disrupted rather than all four and if you have to leave in the middle, only two classes are disrupted. So, there is a way in which it allows for some flexibility. You can even be as dramatic and radical as going to a block schedule like they have at Colorado College or other schools where they have one course at a time. That would be more work for our faculty and may not work as well, but I did like the idea of thinking, “Okay, let’s just prepare for our face-to-face courses to be seven and a half weeks as an institution.” And then it’s the opportunity to experiment with that kind of pedagogy anyway, because some schools have May terms and other things. And so we are not, at Wake Forest, certainly planning that, but it is an interesting fun thought experiment to think about.

John: One issue that we’re talking about on our campus is how faculty should administer final exams, and grading and assessment, and there’s a lot of concern over people trying to give timed exams and put other limits on students. What are your thoughts on how we should deal with assessing students as we move towards the end of the semester?

Betsy: Yeah, that’s a great question. So, there are a number of issues at work here. One is the challenges the students have at home and thinking about good universal design principles of giving students as much time as possible if it’s not one of your outcomes. If doing things quickly is not one of your outcomes, that’s an important thing to think about. Often, also, what’s in the mind of people, though, is academic integrity. And so part of the concern of a number of faculty is: “Well, I’m usually proctoring it in person. So, how do I give an exam in a way where I’m not going to be there in person?” And then that raises all sorts of interesting challenges associated with technology and the privacy concerns with those online proctoring systems, and so certainly we’ve been thinking a lot about this too, and how to give advice. One of the first easy answers that anybody who’s in pedagogy is going to say, is come up with different designs for your assessments. And I think, absolutely, we should start there. I don’t need to give a timed exam in my course, there are ways I can write the question where I’m not worried about academic integrity issues. So, there are certainly ways in which that’s possible, but I do want to be mindful of my colleagues in intro languages, or my colleagues in intro math, where there are some recall outcomes that are really important for them. And so I think I always want to just be careful to not say, like, “Oh, how dare you have any recall outcomes because that’s just not good pedagogy.” I don’t think that’s necessarily true, so for those colleagues who have recall outcomes, it becomes a more interesting question. On our blog, we have a post on this that we could share, if you’re interested, where my colleague, Anita McCauley, who’s amazing, posted a flowchart of ways of thinking about, “Okay, if you have an exam, what are ways that you can think through how to do something differently?” And one of the first parts of that flowchart that I really like is, if you’ve already assessed it before, you may not need to assess it again, and so particularly for my colleagues in Spanish and other intro languages, maybe they’ve already assessed their ability to conjugate verbs. Do you need to have it on the final in a cumulative way? That was just something that often has not been on the table and talked about and I think it’s worth saying. But, beyond that, they might be somewhat different outcomes where they have to recall but then explain why the verb was conjugated in that way, and so there are ways you can see whether they know it or not, that they can’t just get on the internet. And so being mindful of the challenges there but also saying that “Let’s try as hard as we can to come up with alternative assessments.” Then the questions of how much time to give them, again, always come back to say, like, “Is speed one of your outcomes?” and almost always it’s not; almost always the reason there are timed exams is because they’re in the timeframe of the class. So there’s 75 minutes for them to sit in the class and take the exam and that’s why there’s a limit. It’s not because speed is actually an outcome. So now they actually have some more flexibility where they could give them more time and the technological tools allow them to give them more time, and you can extend it as far as you want. I will often say, instead of giving accommodations to a student to get extra time, give the whole class extra time, especially as they’re learning new technology. If folks are still committed to traditional recall exam and worried about proctoring…. we, for example, at Wake Forest,have not bought proctoring software… and we’re not using it for a variety of reasons, and so one of the things I recommend is if you absolutely are still committed to that, then you can do a synchronous session just like you would normally where they’re taking the exam, and it’s you, not some outside vendor or AI etc, as you would in the classroom.

John: I’m not sure if the problem though, is just due to recall type exams because I can speak from my own experience. Last week I gave a test which were all applications in econometrics and copies of the questions (where there were many different variants for one problem, there were seven variants), most of those problems ended up on Chegg within about 15 minutes of the release of that, and answers were posted. Many of them were really bad answers, which helped make it really easy to find these things…

Betsy: [LAUGHTER] Yes, yes.

John: …within less than an hour of the time the exam was released. So even when people are doing some problems, there are some issues, or even when they’re asked to write essays, there are people out there who are willing to provide those responses for them.

Betsy: Oh, yes. And actually, that will be true in face-to-face classes, too, if you’re not doing in class essays. That’s the one level of academic integrity that you just are never going to be able to catch if you pay somebody to write your essays for you or take your online exam. My background, incidentally, is as an ethicist, so I think a lot about questions of academic integrity. I always get mad at my students when I give this lecture like, “This is an ethics class, you need to take this seriously.” But it is true that the empirical research on student behavior in this regard is not heartening. Let’s put it that way. So I really appreciate all the literature about “We need to trust our students,” and there’s a certain framework of what happens when we come into a course where we don’t trust our students, but the empirical literature about what students admit to have done is really not heartening, and so I do think it’s okay for us to think about these questions that you’re thinking about, John, which is, “Okay, we’re creating conditions where they’re tempted,” and that’s something also we don’t want to do either is to create the conditions where students might be tempted, particularly for students who do have academic integrity, because then they’re at a disadvantage if they choose not to engage in that kind of sharing of resources. What did you do, John, how did you address this?

John: I’m just dealing with it now, I was just grading those today.

Betsy: Yeah, it’s tough.

John: So right now I’m trying to identify the students and I’ll be having conversations with them. Because there were so many varieties of questions out there, it’s going to be pretty easy to identify which student did which. One interesting thing is, someone took one of the answers and ran it through a paraphrasing tool so that the “error terms” in the equation became “blunder terms” in the equation, which was a pretty obvious paraphrase. It was interesting.

Betsy: One of the things I’ve appreciated about this moment and having conversations like you and I are having right now is that it’s encouraged some faculty to think in different ways about assessment. They have a standard way of assessing “This is the kind of thing that I’ve done for years,” and now I have to think, “Oh, what could I possibly do differently?” So one thing I just keep coming back to, when I think about my own courses… there are challenges with this… there are problems with this, because it can be stressful for students… but I think oral exams are often some of the most effective ways to see whether a student knows something is that you, face-to-face, come to my office hour, and let’s talk about it. Tell me, and I’ll ask follow-up questions. That’s a way to really tell whether a student knows something, and so you can still do that virtually. Now that takes more time, especially if you have a big class, but thinking sort of outside of the box in that way of “How can I verify?” is important. I have a couple colleagues that are ethicists too, who have devoted their life to this issue of academic integrity and it consumes them. In some ways, I understand that, because it’s a real violation of trust, and it harms other students. But at the same time, too, I worry sometimes that it becomes so consuming for us that we lose track of all the other things that we should be thinking about with teaching, and so, in this scenario, where it’s as crazy as it is, this is why I think the pass-fail designation that many of our schools have done have made things easier, because we also know empirically that students are less likely to cheat when it’s a pass-fail environment. I think the fact that many of our schools did optional pass-fail means that we’re still in this wrinkle space where many of our students still want to get the good grade, and so they’re taking it for a grade and there’s still temptations. But thinking of ways to make it less high stakes can be another way as well to reduce the likelihood of academic integrity, but it is going to be a challenge that there’s no quick and easy solution for. I don’t have your solution, John.

John: Well, I don’t either, right now.

Betsy: Maybe somebody will… that they can tell us too, who listens to this podcast.

John: One thing I am also doing is I have scaffolded assignments where they have to develop things from the very beginning up to their final projects, and there it’s much more difficult for academic integrity problems to show up because they’ve been guided and getting feedback all the way through and that tends to reduce it, but when you’re trying to test some other things that they’re not using in their projects, but might need to know in the future, there are challenges there.

Rebecca: I think another question that’s come up quite a bit is how to grade fairly just over the course of the semester, either this semester or a future semester when there might be potential for another outbreak or something, when students are not in optimal work conditions, there’s distractions, they might be sick, they might be dealing with family members who are sick. So what do we do to make sure we’re fair?

Betsy: Again, coming back to me as an ethicist, I think a lot about academic integrity, but also about grades and what it means to be fair, and there’s some people who would make the argument that there’s certain notions of fairness… that it’s impossible to grade fairly, even in normal situations, especially if we’re taking into account differences in student background, etc., that they’re always going to be disadvantaged students in our classes. And so thinking about what a grade is, is really important. And again, I’ve been heartened by the fact that these challenges have led so many of our faculty to start thinking in new ways about “What the heck is a grade and how do I want to think about my grades?” And I do think that one way of thinking about fair grades is actually not the model of “Well, we need to take account of all these challenges the students have,” one way of thinking about their grades is that all the grade is, is a measure of their performance. Now, you could say that that’s unjust for other reasons, but that it’s at least I’m treating all the students the same. So this is maybe the difference between equality and equity. So like we’re treating them all equally, that’s a measure of performance and mastery, so it’s ensuring the integrity of the grade. But what’s interesting is that most of us don’t actually grade that way. Most of us have all sorts of other things in our grading scheme that are about behavior, rather than about outcomes. So like, “You have to show up, you have to turn these in by this due date, you have to make sure you participate in class,” and I have those in my typical grading scheme as well, and those we’ll refer to as behavioral grades. And there are some educational theorists, as you two probably know, that would argue that you should never grade on behavior, you should never have behavioral grades. I think we could have a much longer discussion about this. I sort of think there are some good reasons for doing it in the context of higher ed at least. But I think in this scenario, this is if there’s any scenario and this is what I wrote about in one of our first blog posts, if there’s any scenario where that would be unfair, the kind of behavioral grading, it would be this scenario because some of our students did not choose this, they’re in different time zones, they can’t make it to our class, they have to deal with things at home. They were already in the midst of the course too, so it’s not as if we say, “Well, wait a year and come back to us when you’re ready to take the class fully,” because they were ready, and we kicked them off campus. So there is all sorts of other complications here to the traditional model of like, “Well, wait until you’re ready to take a class.” They can’t, they were already enrolled, they already paid, we’re not giving them refunds. So in this context, being as accommodating as possible, and making our courses as accessible as possible, is really important. And some people have even argued, this is why we should give them all A’s like some people have argued, not just pass-fail, but actually all A’s would be a better approach. Because to say like, “Look, you’ve done some work this semester, let’s move on and give you all A’s,” of course that creates challenges for some of our colleagues, who are going to say “What about the integrity of the grades for future courses? Is that fair to students who take it a different time and don’t get the A?” So what I have argued for, but it’s again, not a perfect solution, is really dropping any behavioral grades that you have in these scenarios, at least for this context, and then really focusing on your mastery outcomes, but also being reasonable about the number of outcomes students can master in this scenario. So I actually dropped two outcomes from my course completely, completely dropped them. Now, that’s easier for me to do in an intro religion class than it is in an intro calc class where they’re prepared for the next course. So I always want to be mindful of the differences of my colleagues in different disciplines. But, if you are able to drop outcomes, you can drop them and still be rigorous with the outcomes you still have and being a little bit more compassionate and sensitive to your students. But doing mastery based grading also can be helpful in the sense that, for me, students get multiple shots at showing mastery, and so this would be like specifications grading if you want to read more for the fall, so they have multiple opportunities to show. So, if they have a bad week or an assignment doesn’t work well, they can try again, and as long as by the end of the semester they showed mastery, that’s enough. It’s not about averaging over the course of the semester, and so I already had a mastery based grading system in my course before I began this semester, so I wasn’t recommending to people in this transition, “Oh, completely revise your grading scheme,” that would be not helpful. But if people are thinking about the fall, you know, it might be worth considering thinking about that. There are downsides to mastery based grading too, so I don’t want to act as if it’s this, like, solution to everything, but it might be worth investigating a little bit and maybe incorporating some aspects of mastery based grading into your teaching.

John: And we did have an earlier podcast episode on specifications grading with Linda Nilson.

Betsy: Oh, wonderful.

John: So we can refer people back to that in our show notes as well.

Betsy: Yeah, it’s wonderful.

John: One of the issues with equity, as you mentioned, is that that problem became, I think, much more severe when students were suddenly sent home. On campuses, at least there’s some attempt to equalize that, that everyone gets access to high- speed internet. There’s computer labs in most campuses spread out across campus, and we also don’t have as much of an issue with food insecurity, at least for our on-campus students while they’re there. Suddenly when students are sent home, all those things disappear and the issues of inequity, I think, become a whole lot more severe and it’s something, as you said, we need to be much more mindful of.

Betsy: Yeah, one of the things that I really appreciated from Tom Tobin’s book on universal design, he has a distinction. I don’t actually know if it’s his… it might be his or it might just be generally in the literature on universal design… is distinguishing between access skills and target skills that you want your students to learn versus things they have to know or be able to do to access your material. What I really appreciate, as it helps us think about something as simple as like having a good internet connection, that should not influence their grade, because it’s not one of our target skills. That’s not what we want the grade to be reflecting, whether they had good internet connection. What we want the grade to be reflecting are the target skills that we’re interested in. So I think the way to think about equity here is to focus on any place where things that are irrelevant to your course outcomes are getting in the way of students being able to learn and demonstrate their mastery. That’s where you want to be lenient, that’s where you want to come up with solutions. So for example, in my first-year writing courses in English as a second language, if the thing that you’re assessing is not grammar, if the thing you’re assessing is the way they develop their ideas, the grammar can be a barrier. So there are ways in which you don’t want to grade on that, because your target skills are really about developing ideas. And so that’s a sort of inclusive teaching practice that’s really important in this scenario. What are the things that make it difficult for the students to show up in our Zoom session? And how am I going to create alternatives for them? One thing that we have suggested to our faculty is if you’re doing Zoom sessions, of course, they should be optional. But we also suggested recording it. So the students who couldn’t be there could watch it setting aside the problems with privacy, of course. We can talk about that too. But there’s another wrinkle there too, which is that then that means some students get the interactive, quote, unquote, face-to-face engagement, but the other students only get to watch recordings the whole time. So, one of the things we’ve also said is for equity is also to think of other ways you can engage with those students who can’t come to the Zoom sessions in a way that’s asynchronous or that perhaps at a separate time without overly burdening the faculty member as well.

John: One of the things I’ve done is I’ve shared my cell phone number because all the students have cell phones. [LAUGHTER] I’ve only done that once or twice before in senior-level classes. But this time I’ve done it with all my classes. I did get a phone call coming in right at the beginning when we started recording, and I sent back a text saying I’ll contact you later. But, that has helped because some students do have issues with being able to use Zoom.

Rebecca: And there are certainly tools that you can use to allow you to provide a number that’s not your actual cell phone number that students can still use your phone or texting to communicate.

John: You could use Google phone or you…

Betsy: There’s another one, though, that I’ve used in the past maybe five or six years ago, it’s used in K through 12 environments. Oh, Remind… Remind is the one. Yeah, so I actually used that when I started to realize my students weren’t checking email anymore. I was like, “Oh, this isn’t gonna work, emails not going to work anymore. So I need to find some other way to connect.” But that’s great to be as accessible as possible to your students, but recognizing also that equity issues for us, as faculty. Some of my colleagues can do that more easily than other colleagues who have three kids at home that they’re homeschooling. And so that’s a part of the challenge of this scenario as well, it’s not just what we know is good teaching practice, but also the labor implications for faculty too… that are significant.

Rebecca: Following up on that, that’s a really important consideration is the balance of fairness between both faculty and students because it’s certainly not a situation that any of us signed up for, but we’re all trying to manage. And it’s really possible that we might be in a similar situation in the fall, maybe not exactly the same in that we’ll have a little warning, but it still could happen. So, how do we think about balancing the ability to pivot and make sure that we’re thinking about the ability of teaching remotely without getting too much burden on faculty, but still have really good learning opportunities for students?

Betsy: Yeah, that’s a great question. So I think part of it is trying to think about efficient ways of development. So I was, and you, too, may feel the same way, that that week that we had to transition, I have never seen so much learning happening in a week and so much effort and work and those of us in faculty development probably would never have dreamed. I mean, I don’t know, maybe we would have dreamed that that would happen, but it was a really remarkable thing to see that faculty teaching other faculty can accelerate this in a way that often the model of one instructional designer with one faculty member for six months, that model like that’s how much we need, well, maybe not, now. Maybe we see that if you have to get it done, we’ll get it done, and we can have one-to-many trainings, we can have faculty training each other, we can accelerate that, in some ways, I think is important. But support is also important. So, making sure that we’re supporting faculty as they’re learning what things they can do, and also what we often do in faculty development is talk about efficiencies. So, it’s not just “We’re going to give you a million new pedagogies that we know work, but we’re going to give you a pedagogy that’s actually going to save you time,” and that is really powerful with our faculty and I think we can do the same thing here. So, if we know that there’s a faculty member who has children and has had a hard time with this transition, because they can’t do synchronous Zoom sessions, maybe we talk with them about other alternatives that might be easier for them that they can prep in advance, that will make that transition easier without having to show up at a set time for those synchronous sessions with their children at home. So, it doesn’t solve it, but I do think we should work really hard to come up with the most efficient ways of making the best outcomes possible given the resources that we have. And I think adjusting resources… so we’ve talked about at Wake Forest, outside of teaching and learning, some of our staff, their jobs are no longer really needed, so let’s transition them to other places where we need support. I think you could do the same thing with faculty as well. So, maybe those who have the capability of teaching more or have taught online before, maybe they do more in the fall, but then they get a leave in the spring. There are ways in which you can move things around. Again, I’m not a Dean making these decisions, but being creative about making sure to share the load equally. One question that has come up here, which is really interesting, is that for our faculty who teach more as part of their load, in some ways, this is certainly harder on them than those who have a more balanced teaching and research pipeline, because most of the effort here is in revising courses. Of course, if you have a lab that you have to shut down, certainly that’s a lot of effort. But, making sure we’re mindful of the differential impacts of this transition on our faculty and figuring out ways, not that we’re going to pay them for it, but figuring out ways that we might be able to balance the load moving forward once things go back to quote unquote normal, if they ever do go back to normal [KNOCKING SOUND] …knock on wood here.

Rebecca: I know one thing that I’m thinking about, having small children, is that I’m thinking about all the things that require a little less cognitive load that I’m doing right now while I have a toddler at home, and then when I think I’m going to have daycare again, I’m going to take advantage and do the things that actually require a lot more cognition. And I’m planning to do those at those times, including things like recordings or things like that, that I know I might need to do just to have it in the wings just in case something happens in the fall.

Betsy: I think this is an opportunity for all of us in higher ed to think creatively about how we distribute workload and how we think about the semester and timelines. So, even before this happened, our team read the book Deep Work, and we were just talking about how to create space in our daily work to do intensive deep work, and one of the stories he tells in the book is about a faculty member who stacked his courses so that they were all in one semester. So you know, you have a two-two load or three-three load and he decided to do six in one semester, and then none in the next which normally that sounds crazy, but there’s a way in which that could be really helpful in certain contexts and I think this is an opportunity to think about that. So, those who are doing really intensive work, building online courses, maybe they do a number of them, because it scales, economies of scale, like they do a number of them in the fall and then in the spring, they don’t have to teach… you know, ways of thinking about how to balance this, and then it also would allow us as faculty developers to work with a smaller cohort of faculty, rather than having to work with every single faculty member. Now, I don’t imagine we’ll do that, but it is an opportunity to think of these creative ways of making the workload more equitable as well.

John: And faculty, as human beings, tend to keep doing things the same way as they’ve always done them until there’s some sort of disruption. This certainly has been a fairly substantial disruption, and I think a lot of people, as you said, have learned how to use new tools and at least from what I’ve been hearing, many people now having discovered using Kahoot for quizzing, for example, or using Hypothesis. I’ve been giving workshops on Hypothesis for a while on campus, but not many people adopted it. All of a sudden, I’m getting all these questions about using Hypothesis where people are using it for peer review of documents, where they’re using it in the LMS or more broadly, and I’m hoping that this will continue in the fall. What sort of reactions have you been getting from faculty who are trying some new tools?

Betsy: Yeah, I mean, there’s so many who have said, “Oh, wow, I can totally use this in my face-to-face classes,” and that’s really exciting to hear, that they’re gonna keep it, they’re gonna keep the strategies in their face-to-face courses, or if it needs to go remote, of course, as well. As well as, “Oh, now I know how to use Canvas, so I’ll actually use the gradebook.” Things that are going to be nice for our students as well. Students have been asking for to have a place where they can see all their courses together. I think there was a kind of fear about these technologies in some ways and now that they were forced to do that, “Ah, it’s not so hard.” Now some things are difficult, some things are challenging associated with developing a really well designed online course, but some of these little tools that they have to use in this environment can be helpful in what they’re traditionally doing in their face-to-face courses and I’ve seen many of them say they’re going to do that which is such a wonderful thing to hear as well as pedagogical decisions they’ve had to make about assessment, about universal design, about academic integrity, grading, all the things they’re learning there can also translate back to their courses too, even if we don’t go remote.

Rebecca: And I think all those like crossover areas are ways that faculty can be more nimble. The word pivoting has been used a lot, but I think also being nimble, “I’m using this tool or method and it works both online and in person, so it doesn’t matter which modality I’m using” is something to think about. I did want to just ask one last question related to grading and evaluation, and that’s about motivating students to achieve our learning outcomes when there are so many other things in the world right now that we might be thinking about.

Betsy: Yeah, that’s a great question. I often like to quote the former Secretary of Education that said, “There’s only three things that matter in education; motivation, motivation, and motivation.” [LAUGHTER] So motivation is super, super important when we think about how students learn. We can design the coolest evidence-informed course and design, but if students aren’t motivated, it doesn’t matter. So thinking about our student contexts, and their motivation is really central to their learning, let alone how we’re going to grade them. And so there are a number of things we know that lead to motivation. Sort of important is the students have a choice and that they have some agency or ownership over what’s happening, and so I know a lot of my colleagues at Wake Forest did this and I did as well, is when we made this transition is to ask the students, “So what’s going on with you? What’s your preferences for how we restructure the course? How would you like to learn moving forward?” and to keep being in conversation with our students. And what I did, for example is, now again, I had the flexibility to do this in a religious studies course, but I basically threw out that project at the end of my semester, and so instead had time to say, “What do you want to read about? What things about religion do you want to know about?” And so we’ve been reading about religion in violence, religion in COVID-19, just things that they’re interested in, and that has allowed me to help a little bit with motivation is to just engage with the students a little bit more, but it’s tough. Typically, I think a lot of times we think of there’s carrots and sticks related to motivation, so you can certainly use sticks if you wanted to with grades, but that often has unintended negative consequences. So the more you can do carrots, which would mean thinking about what do they want to learn. I also think that my students, at least at Wake Forest, really miss each other. It’s a really communal place and they really miss each other, so creating opportunities for them to engage with each other, even if I’m not there. There are lots of little interesting activities I’ve seen people suggest where they get together and have video chats in groups, and then record them for the professor. Creating opportunities for them to spend time with one another… They will just want to spend time with each other, whether it’s about learning or not, but if you sneak in the learning, that can be something that will motivate them too, but the reality is some of our students, there are too many other more important things on their plate and we need to acknowledge that, and so I’ve tried to make my students feel that it’s okay to say that, that I’m not disappointed in them if they don’t do as well or if they choose to take it pass-fail that like, “Look, this is just a religious studies class. It’s one class among many. There are many other more important things happening right now. Yes, we want to help you learn if you want to learn, if you want to complete the course and get the credits you get, but we all know that there are other things that are taking our attention away right now, and that’s understandable,” and being sympathetic about that, I think, can also be motivating because they’re not demoralized if they don’t do well. “That’s okay. She understands. I’ll give it a try next week.”

Rebecca: I think that humanity piece is key, both for students and for faculty, and it makes people feel like they have a sense of belonging but also that belonging is often motivating.

John: We always end with the question. What’s next? Which is a question on everyone’s mind right now.

Betsy: I think for me, and maybe this is unique to me or my colleagues who are in teaching and learning centers and in faculty development. What’s next is I want to have some time to reflect back on what I’ve learned about faculty from this transition, and what I’ve learned about faculty development from this transition, and we talked a little bit about this in our earlier conversation, but I was really struck by what I saw on that week off that we had to learn how to improve. I mean, again, I need to spend more time thinking about this and what we’ve learned, but one of the things that was really striking to me was how important having a dedicated time to talk about teaching was, like, “This is a week where you’re going to work on your classes, faculty,” and often we talk about “How do we motivate faculty to do professional development?” We think about funding, we think about course releases or making it enticing in other ways, but my hypothesis that we learned through this transition is that time and dedicated time and a sort of cultural commitment to saying we’re going to take two days to focus on our teaching. What if we did that every year, and there are some schools that have a faculty development day, but what if we took three days every year where everybody got together and talked about their teaching. And I think that’s just one example of something that I would like to reflect on, but I think there are many other things that have happened in the past three weeks that can help inform the way we think about faculty development and I’m really excited to think about that as we, as a center, think about how we work with faculty going forward.

John: Things like that Facebook group that you mentioned, and we have a similar one, has been really helpful in building more of a community than I’ve ever seen before.

Betsy: Absolutely. Yep, I completely agree.

John: Well, thank you. This has been fascinating, and we wish you luck.

Betsy: Thanks for inviting me. It was great to talk with you all.

Rebecca: Yeah. Thank you so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Savannah Norton.

[MUSIC]

127. Gender and Grade Changes

Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, Dr. Cher Li joins us to discuss these gender differences in grade change requests in college and why they might occur.

Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes.

Show Notes

Transcript

John: Grade change requests in college are relatively rare, but when they do occur, evidence suggests that male students make the request more often than female students. In this episode, we discuss these gender differences in grade change requests in college and why they might occur.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Cher Li. Cher is an assistant professor of economics at Colorado State University. Her research focuses on how public policies and social institutions affect the decisions of, and outcomes for, women. She is also a co-author of a January 2020 National Bureau of Economic Research working paper that investigates gender differences in grade changes. Welcome, Cher.

Cher: Thank you. It’s great to be here.

Rebecca: Today’s teas are:

Cher: I have Oolong tea from the Ali Mountain area in Taiwan. It is one of the best tea regions in Taiwan.

Rebecca: Sounds good.

John: Nice. And I have Cranberry Blood Orange black tea.

Rebecca: And I have a mango black tea today. John’s looking at me a little strangely because it’s very unusual for me. [LAUGHTER]

John: We’ve invited you here today to discuss your recent working paper with Basit Zafar on regrading requests and outcomes. Could you tell us a bit about the study?

Cher: Yes, this study is about gender differences in grade changes in college, and we look into the differences and why are the causes for the gender differences?

John: What prompted you to look into this?

Cher: So this research is actually motivated by my personal experience. In the last six years, I had many male students, but only one female student asking me to bump up their grade at the end of the semester. And I know from previous studies that men are more likely than women to negotiate for their salaries and promotions, and that made me wonder if these kinds of differences actually started earlier when they’re still in college. So, that is the motivation for this study.

Rebecca: Can you talk a little bit about how you conducted the study?

Cher: Sure. We tackled this research using three very different sources of data, including administrative records from a very large four-year university and we also use surveys and laboratory experiments. So, the first data source is a very unique administrative data from Colorado State University between 2010 and 2016. We have over 1.3 million observations of grade changes across all the departments, and this data set is very unique because it recorded not just the final grade, but it also recorded any changes associated with it. So we focus on the changes made by instructors, and our analysis showed that, although grade changes are pretty rare events, 95% of the time they will change to a better grade, and among those positive changes, men are about 18.6% more likely than women to get a better grade. And the difference in grade changes cannot be explained by the characteristics of students, instructors, and classes. However, the administrative record has several limitations. First of all, it doesn’t provide information on the cases when any regrade request was rejected by instructors, and it also doesn’t tell us about the grade changes during the midterm or any assignment throughout the semester. And finally, it doesn’t tell us “What is the scenario that caused the gender difference in grade changes?” So we can think about at least three different scenarios that all lead to the same data pattern. For instance, the first one is male students are simply more likely to ask, although instructors treat men and women equally… so, that is a possibility, given the previous literature evidence. And the second potential scenario is that male and female students are equally likely to ask, but somehow instructors are treating men more favorably, and if it is this scenario, we’re going to have something to worry about about discrimination. And the third scenario we thought about is, there may be a difference in terms of the timing of asking. So if men procrastinate while women are more organized throughout the semester in they ask their instructor to make changes while they find any errors during the semester, then they have no need to ask at the end of the semester. So, to examine the scenarios, we conducted two surveys, one on instructors and the other one on students, and we checked whether the reports are very similar across different surveys. And surprisingly to us, both surveys show pretty consistent patterns: that male students are indeed more likely to ask, and we find no evidence that instructors are treating males students more favourably. And we also didn’t find any support that there is a timing difference in terms of their asking.

John: That’s certainly consistent with what I’ve observed myself, that males have been more likely to ask both during the semester and after.

Rebecca: I don’t think I’ve gotten that many requests, generally.

John: Well, I haven’t either, I may get one a year or two a year out of like seven or eight hundred students, but…

Rebecca: I can’t think of a single one. I know that I have but I can’t think of a particular…

John: Well, your classes are smaller, too. So, that could be a factor.

Cher: Yeah. So I also typically only got like one to two students per semester at making that request. So it jumps out at me when there is only like one female student throughout the whole six years of my experience, that really makes a difference to me when I compare the frequency in terms of they approach me at the end of the semester.

John: One of the nice things I liked about it was your incentivized controlled experiment that looks at why there may be differences. Could you tell us a little bit about that aspect of the study?

Cher: Definitely. So focusing on the gender difference in asking, we did this lab experiment to answer that question, in particular in the lab setting. The participants basically take a quiz of 20 questions, and their payoff is dependent on the grade in the quiz. So we tell them we’re going to give them the grade. However, we also tell them the grade is noisy. So, we randomly picked three out from the 20 questions to randomly grade them. The grade we tell them could be their true grade, but it could also be higher or lower than the true grade. And then they have to decide under 10 different cost scenarios, ranging from paying up to $3.50 to being paid to $1, whether they are willing to pay the cost in each of the scenarios to get a regrade. If a participant chooses to pay the cost to ask for a regrade, then the true grade is revealed and their payoff is adjusted and we subtract the cost they have to pay, and if a participant chooses not to ask for a regrade, then the initial grade becomes final and they will be paid accordingly. And we find that, as long as the cost is positive, so basically the participants have to make a payment, men are always more likely than women to pay the cost to ask for regrades, and the gender difference in asking cannot be explained by their difference in risk aversion. We find that other majors, including their difference in confidence level, in their uncertainty about their performance, and personality traits explain nearly half of the gender differences in asking.

John: How accurate were the students’ metacognitions in terms of how appropriately they made a decision to request regrades? Did they gain by that on average, or did they lose?

Cher: So in general, they have pretty good guesses about their expectations in terms of how well they do in the quiz. However, their decision here really depends on their attitude in terms of pursuing more aggressively in the regrade or not. And we find that, in general, most students who decided to pursue the regrade were likely to get a better outcome, and we find that depends on the cost, basically. So when the cost is low, the more likely a student will get a better outcome, because they are facing a very low cost to offset the gain they got from a better grade. And the reason that they are more likely to get a better outcome is due to our lab experiment design, we make it asymmetric, basically. So, we are trying to mimic the real world scenario. So from the administrative data, we find that 95% of the time, students got their grade change to a better grade. Only 5% of the time it got changed to a worse grade. We intentionally designed our lab experiment to allow that asymmetry. So basically, when a student asks, they will have a higher chance to get a better grade than to get a worse grade.

John: How much more common was it for males and females at low, and then again, at high prices?

Cher: When there is a cost that really they have to pay out of their own pocket, around like half of male students are willing to pay the cost, while only about one third of women are willing to do so. However, when the cost is zero or they got paid to do it, it’s very similar. I think their percentage of requests is very high. It’s like nearly 80% when they were getting paid $1 to do so.

John: What happened when the cost was high? I believe you found some interesting effects for males at that.

Cher: So when the cost is high, we find that the gain from asking actually disappeared, because now the high cost offset the gain made by men. So although men are more likely to ask, they are not always made better off, so they are only gaining relative to women when the cost is relatively low. When the cost is above like $1.50, we see that men are also more likely to see their payoff decrease compared to women.

Rebecca: Can you talk a little bit about the gender difference in the regrade requests and what might explain those differences in general after looking at those three different approaches that you’ve taken to study it?

Cher: So we find that, it seems to be, there are some subjective beliefs that are different to begin with. So first of all, there are some evidence from other studies that men and women are different in terms of confidence level, and men tend to be more overconfident than women in a lot of contexts. So, we explore that too in our study, and we find that indeed, when we look into each of the questions they answer, we also ask them how confident they are on each of the questions. And we find that when they got the answer wrong, men and women are similarly over optimistic about their accuracy. However, when their answers are actually correct, women are much less assertive about it, so they are less confident about their correct answer. So that is part of the reason that is driving the gender difference in asking, and the other factor that we found are important in terms of the gender difference in asking is about their perception about the potential outcome, so basically the uncertainty they face when they make that decision. So we asked them about like, “What is the number of questions you think you got right?” And they will give us their best guess. And then we ask them then for that scenario, “What is the likelihood you think that’s gonna be happening?” So they will tell us like, oh, if they are very confident about “I’m getting 15 questions right out”, maybe assigning 100% as the probability for that outcome, if I’m super confident that is the outcome. But then if I’m not very confident about my guess I maybe say “Okay, 50% probably,” then I’m not really certain about that outcome. So we find that when men and women are giving us those guesses, women are much less certain about their guess. So they are likely to think it could be different outcomes, maybe it’s 14 or 13 questions they got it right, so they would also assign the probability of other outcomes that they anticipated. And that also explained a very large proportion of the gender gap, and we also find there are some differences in their personality traits. For instance, women are more agreeable than men, so they may be more agreeable to the grade they received and less likely to challenge it in terms of the context of asking.

Rebecca: When we start thinking about the impact this might have on GPAs or their academic standing, how does this really impact men and women differently?

Cher: So the overall impact on GPA depends on how we measure it. For those people who ask, we did a back of an envelope calculation and we find that for the female students, the accumulative effect from asking is up to .43 points in their overall GPA. But for male students, the number is a bit higher, they could receive up to .47 points as an improvement in their GPA, conditional on asking.

John: Which is a non-trivial difference in terms of that, especially if you were also to extrapolate and think about the possibility that this is also occurring during the class.

Cher: Yeah, we take that into account in extrapolating, and that is a cumulative kind of effect we anticipate. If a student reports a fraction of courses they asked for a regrade, that would be kind of cumulative effect overall.

Rebecca: Do you have any idea of what the not asking is doing, maybe to women, if they’re less likely to ask in the first place, how that might be negatively impacting their GPAs?

Cher: So in terms of not asking, you are comparing to those who ask, so that raw difference could be low depending on who we are comparing to, or we can compare it to those who actually are successful throughout their time, then that difference will be up to like .4 to .5 points in terms of the GPA.

John: At the start of this, you talked about how there’s a lot of evidence out there that women are less likely to bargain for higher wages for promotions and so forth. From those studies, is there a lot of evidence of similar types of causation?

Cher: So most of the studies we saw before look into their different tendency to ask, and one of the particular research I think is very interesting is in terms of the context. In the study made by other economists, including John List in University of Chicago. In their study, they did a field experiment, and they experiment the job applicants’ attitude in terms of negotiation for their salary. In their experiment, there are two settings, one is the salary is ambiguous if there is no hint or any information about it. And then they found under that context, men still negotiated for their salary, but women in that context tend not to negotiate and instead, they signal their willingness to accept a lower offer. In the separate setting, when they make it clear in their advertisement that the job salary is negotiable, then there is no gender difference in terms of asking; men and women are similarly likely to negotiate their salary when that kind of information is clear, that the social cue is you are able to negotiate. In that case, there is no gender disparity, and women are similarly likely to ask. So we find it relevant in our context as well because a lot of time professors do not put a clear policy about regrading, so students may have to take that into their own guess, whether this is an appropriate behavior to make and approach their instructors about it. So I found that kind of ambiguity is relevant in terms of the context when the grade change policy is ambiguous.

John: I’d be really hesitant to put in a procedure explicitly for negotiating grade change requests, but I could see how that could make a difference when it’s normalized as an appropriate policy. But are there any ways that you can see, based on your results, where we could try to compensate for this, where we could try to offset this difference in grade change requests between men and women?

Cher: So in addition to making a transparent regarding policy, I’ll be arguing, first of all, we are not trying to push women to over asking, basically. And a lot of time, we also find in some other literature, they find when you pressure women to ask, the results are not always favorable. So that is truly not our intention. The point we make about transparency in terms of regrading is to make that kind of policy more formal, and sometimes maybe also imposing some cost in terms of the regrading in terms of like, the professor may be able to regrade the entire exam or the assignment, typically, would be, like, not enforced, when students are just bringing your attention to one particular question they think they should get more points and argue for it. But if they have to come through a particular procedure, for instance, making those requests through like documentation, or they have to make some arguments for the points they think they should earn, it would make it clear to students in terms of what they have to jump through, and then other policy that we think may be helpful is in terms of providing better signals. In terms of the performance, for instance, “How is the grade assigned? What is the metric? What is the rubric that a professor uses to assign a higher grade when the student has performed well?” and just really to show them like these are the guidelines why a student would get an A instead of getting a B. What are the distinctions between the different grades, for instance, to make it clear to students why they’re getting their grade, especially when the grading is based on something that’s more ambiguous, like essays or something.

John: So if you grade essays, make sure you provide a rubric that provides a clear delineation and makes a grading process more transparent to everybody.

Cher: Exactly. So when the women are getting a grade and they see, for instance, “I’m getting a B, but I’m qualified for getting an A based on the rubric,” then I will be more confident that I am eligible to ask the professor in terms of whether my grade is accurate or not.

Rebecca: I think it’d be interesting then, the next step would be to see if it’s not just gender differences in asking, but then also, if it’s students that grew up in lower income families or other kinds of situations that might be less likely to ask in the first place.

Cher: Yeah, I think that would be interesting, but we do not have that much information about student income, so we are unable to really look into it. But we do look into some of the different demographics, including their race composition, and we do not find much differences when we account for those differences. And it doesn’t seem to be the main reason that’s driving those gender differences, but there might be some other interesting questions that could be asked in addition to the gender disparity.

Rebecca: One of the things that always comes to mind is if you start putting in policies or things that make it more transparent, it still means that a student would have to take action in some way, so they would have to feel empowered to be able to take those actions. So it’s always curious to kind of think about those power structures beyond just gender.

Cher: Right, I agree.

John: With providing something in a syllabus for challenging grades would probably help reduce those based on what you said.

Rebecca: Especially because it establishes that behavior as something that would be appropriate in that context.

Cher: Yeah, exactly. Because when we ask students in our survey, “Why don’t you ask?” because we start asking them whether they ever consider asking their instructor to reconsider their grade at some point, and if they do, we follow up in asking for the courses they consider to make that request and the reasons they decided not to pursue it, and we find that a non-trivial fraction of students actually just never thought of it. [LAUGHTER] So they don’t think that is appropriate or that is available, so that is one of the reasons we believe it’s good to have a transparent policy in place.

Rebecca: So we usually wrap up our conversations by asking “What’s next?”

Cher: Okay, so the future, so right now our current research is going through the review process, and we have thought about a few directions that we can follow up on the study. One direction is to look into the gender interaction between students and instructors, and we got this question a lot when we presented our work. And a lot of people are curious whether female instructors are getting more requests, and what are the results when you consider those gender interactions? And I think that could be one of a very interesting to look into. And one very interesting thing, I think, from our current study is, when we design our lab experiment, we intentionally take those mechanisms out. So, in our lab design, we shut down a lot of potential mechanisms that could be very important in real life. For instance, the fear of backlash when women ask, which I have found some evidence in current literature, that is totally shut down in our lab experiment. So that is not the reason that’s explaining the gender differences in terms of asking, based on our lab experiment. And we also shut down the gender interactions because the participants in our experiment do not have to interact with a real person. So they just have to make that decision whether they want a regrade or not, and their wish will be granted, basically. And that also shut down any potential gender differences in terms of negotiation skills, if men have been negotiating all along, since they were little, they may become better at it, and that may be contributing to the gender gap in that case.

Rebecca: Those sound like some really interesting follow-up studies.

Cher: Yeah. And we were also thinking about, one very cool thing that could be done, is to follow up with the students into their labor market experience and see how their negotiation behavior may actually lead to some differences in their labor outcomes. Are people who tend to negotiate their grade in school are getting better outcomes when they are working for others, for instance.

John: And I know many alumni programs collect data on first salaries out of college, and that might be an interesting place to do that linkage since you have the administrative data there.

Cher: Exactly.

John: Excellent. I really enjoyed your study, and I like the way you designed the experiment to elicit some detail on why this is happening, because the studies I’ve seen, mostly… just in terms of the labor market… just report that this is occurring, but really haven’t done as much, other than the study you mentioned with List and others, haven’t done that much to try to figure out why it’s happening. So I really enjoyed it.

Cher: Thank you so much. We enjoy the research ourselves, too.

Rebecca: Well, thank you so much for joining us.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. Editing assistance provided by Brittany Jones and Savannah Norton.

[MUSIC]

122. Differential Grading Policies

Students generally receive lower grades in STEM classes than they receive in other disciplines. In this episode, Dr. Peter Arcidiacono joins us to discuss how these differences in grading policies across departments can help to explain the relatively low proportion of female students majoring in many STEM disciplines. Peter is a Professor of Economics at Duke University.

Show Notes

Transcript

John: Students generally receive lower grades in STEM classes than they receive in other disciplines. In this episode, we investigate how these differences in grading policies across departments can help to explain the relatively low proportion of female students majoring in many STEM disciplines.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together, we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

Rebecca: Our guest today is Dr. Peter Arcidiacono. Peter is a Professor of Economics at Duke University. Welcome, Peter.

Peter: Thank you.

John: Welcome.

Peter: Thanks for having me.

John: Our teas today are…

Peter: Green tea. I got it specifically for this conversation.

Rebecca: Excellent. I have English afternoon.

John: I’m drinking Spring Cherry Green tea.

Rebecca: Oh, that’s a little switch for you.

John: I’m looking forward to the spring now that it’s 10 degrees outside here in Oswego in January when we’re recording this.

We’ve invited you here to talk about your December 2019 working paper co-authored with Thomas Ahn, Amy Hopson, and James Thomas, on Equilibrium Grade Inflation with Implications to Female Interest in STEM Majors. In this paper, you know that disciplines with harsher grading policies result in higher earnings than the earnings received by those majoring in disciplines with easier grading policies. Can you first describe what you mean by harsher grading policies?

Peter: Well, harsher grading policies… you look at average grades in STEM classes and economics, they tend to be a lot lower than in the other social sciences and in the humanities. In the school we look at, which is University of Kentucky, it’s a third of a grade lower. And that holds true across many other colleges I’ve looked at, be it the Duke University or Berea College, it just seems to be a general pattern that the STEM classes and economics tend to have lower grades. And in fact, there was a very nice book written about this with Duke data called Grade Inflation: a Crisis in Higher Education or something like that written by Valen Johnson a few years back, and it was making that exact point about the big grading differences across disciplines.

John: How does your model explain the grading differential across departments?

Peter: Yes, so that to me is where the big contribution of the paper lies, though. I don’t think our model’s particularly satisfactory in this regard. I think lots of people have noticed in the past these differences in grading standards. Where I think our contribution is, is thinking about how those grading standards impact different groups of individuals, but also in what you just asked about: What are the sources of this? And I think a portion of it is certainly demand for courses. So, at the school we look at, the STEM classes are twice as big as the non-STEM classes. An analogy from my own experience… when I first got to Duke, no one signed up for my grad labor class. And that was a bit of a disaster, because that would mean I’d have to do a new prep for some other course. And what I did is I bribed two students to take the class. And the way I did that was I said, “The class is going to be: we’re going to write a paper together.” And it was a great experience for them. They did get a paper, they had a revise and resubmit when they went out in the job market. And what do you know, they both got A’s in the class, but now I’m a little more popular. Not everyone gets A’s and I’m not co-authoring papers with everyone in the class. So I think part of it is these differences in demand for the classes affect things. You hear about weed-out classes, they’re weed-out classes because there’s plenty of students who are interested in taking that class, so they can afford to do that. If you’re in a discipline that has low demand, you’re not going to have a weed-out class because you’re not gonna have any students. So, I think that’s a portion of it. A portion of it can also be things like differences in preferences, maybe it is the case that some disciplines are just meaner than other disciplines. And a third component may be that it’s just easier to grade those classes, it might be easier to grade a math problem than an essay. I’m a little skeptical of the last one, in part because when I talked to high school students, it doesn’t appear to be the case that the math classes are that much easier than the humanities classes, for example, or grade that much harder than. And I think part of that is in high school, you have to take those humanities classes and when you get to college, you have a lot more choice and that influences things.

John: This also seems to relate to the academic labor markets. Because there’s so much demand for classes, there’s often more demand for faculty. So, faculty are not under as much risk of losing their job if they don’t fill their classes. While in markets in those disciplines where grades tend to be lower, labor markets tend to have a little more slack in them. And faculty perhaps may, using the analogy you used before, have to be a little more generous to make sure their classes fill so they can keep their jobs, particularly when there’s a large component of adjunct faculty and contingent faculty.

Peter: I think that’s exactly right. I’ve heard some economics professors brag about their low teaching evaluations. I just thought they can get away with that in other disciplines.

John: And we see that in many STEM fields as well, at least anecdotally… certainly not on our campus…[LAUGHTER] But certainly many STEM disciplines do have that reputation of bragging about keeping their grades low and having high standards and such things.

Peter: That’s right. You know, in my view, it’s when you change what you grade around, that doesn’t necessarily mean that the standards… how much work is required in the class may stay the same. And that’s where, I think what’s interesting about men and women, is that women actually study much more than men. So, if you change the workloads, that’s more of a deterrent for men than it is for women. So, you can think about different mixes of either giving high grades but with high workloads, or lower grades, with lower workloads and student life, on average being different between those things, but they won’t be indifferent between men and women. At least on average. Obviously, this is just an on average statement. There are some women who are not interested in studying hard, and some men who are. On average, that appears to be the case.

John: And you do know, though, that the STEM disciplines not only assigned lower grades, but they also require larger workloads based on the survey data you were looking at, I believe.

Peter: That’s right, substantially higher workloads. And that was my own experience… that where I went to college, I guess economics hadn’t quite crossed over to STEM at the time, but I started as a chemistry major, and there the formulas are a lot more complicated and you had to memorize them. In economics, I didn’t have to memorize the formulas and they were much simpler than those in the chemistry major and I had to work a lot harder in those chemistry classes.

Rebecca: Can you talk a little bit about how the difference in grading policies impacts underrepresented groups in these different disciplines? So in STEM, for example, where you’re saying there’s these stricter grading policies, there’s less women, there’s less underrepresented groups in general.

Peter: That’s right.

Rebecca: How do those things connect? Can you talk us through that a little bit?

Peter: Yeah, actually I had caused quite a stir here at Duke back in 2011, talking about underrepresented minorities, because you do see, for example, African Americans at Duke… They came in just as interested in the STEM classes in economics as white students, but they left at a much higher rate. And so the negative headline was something like, you know, “Potentially racist study says Black students are taking the easy way out,” and that was not the case. It was that they’re coming in with different levels of preparation, due to pre-college inequalities. We have serious differences in pre-college education. And that’s resulted in this coming in and then finding that it’s very hard to succeed in the STEM disciplines. For women, the story is different. And there, because women come in just as prepared as men and are interested in studying a lot more. For them, it’s not the workloads that are affecting this, but more the lower grades themselves, given that they’re working so hard, and then to see these low grades seems to have a bigger effect on women than men. I think one of my very interesting result from some of the work that Stinebrickner and Stinebrickner have done is actually asked men and women, “What grades do you expect you’re going to get?” And then they’ve showed them their grades later and said, “Now, that didn’t work out quite as you expected.” They got lower grades. Why do you think that was, and men were much more likely to say it was bad luck. And you could see that they actually believed it was bad luck, because they continue to take the same classes where they were getting low grades, and they didn’t change their study habits. Whereas women were much more likely to say that it was something about them, either that they needed to study more, or they weren’t as prepared as they had hoped to be for the class. And then you can see that they ended up being more likely to switch to different types of classes and to change their study habits.

John: I think you also found that women tended to not just equally prepare, but I think women came in with somewhat higher GPAs, if I remember correctly.

Peter: They do come in with higher high school GPAs, for sure, and they get better grades in college than the men, both in STEM classes and in non-STEM classes.

John: So the difference seems to be that women are more sensitive to the lower grades or they tend to blame themselves for the lower grades, while the men tend to be less cognizant of the fact that they may be the cause of the lower grades.

Rebecca: I feel like I’ve been socialized to blame myself for not being adequate.

Peter: Yeah, it’s been remarkable, you know I have five boys. And they’re not socialized that way, despite my best efforts to get them to really care about their grades and to think that studying even matters. Here I am, a researcher who knows that studying matters, and yet some of my boys are convinced that if they do poorly, it’s bad luck. It has nothing to do with their studying.

John: One of the things I assigned in my labor class is a reading from, I think it’s a sociologist, who noted that women seem to be more concerned with what she termed “local status,” while men tend to be more concerned with “global status.” Basically, the argument is that men tend to be more concerned with their income expectations and the impact that the courses in the majors have on their grades, while women tend to be much more concerned with the relative grades and seeing grades as a perception of their worth or their value in some sense, which makes them much more sensitive to that grading differential between majors. While men tend not to be as concerned about their local standing, their relative status, while they tend to be more concerned with how the degree will impact their future earnings.

Peter: And I think it’s compounded by the fact that I don’t think people know when they go to college how different the grades are. I think that they figure it out rather quickly, but at first they just don’t know that they’re graded so differently. I mean by the time they figure it out, they’ve already made the switches, which is too bad.

Rebecca: So what do we do so that there’s more representation in these disciplines?

Peter: Well I think at a minimum, that universities should at least have an obligation to tell people, “Here’s what the average grades are in these different classes,” so that students can make informed choices. I personally am in favor of doing something where, like they did at Wellesley, where they put in something that either capped the average GPA in the class, or having all the classes that are above a certain size give the same median grade so that people are not having their decisions distorted by the fact that one department is giving lower grades than another department. It’s not as though they’re learning less in these STEM classes, just because they’re getting these lower grades. They’re actually working very hard in these classes, so, it would be nice if that was somewhat remedied.

Rebecca: I wonder if even knowing the information across institutions and knowing that people get lower grades in these fields, no matter what school they’re at, would actually be useful to those groups.

Peter: Oh, I think it’d be very helpful. Yeah.

Rebecca: Not just within your own institution.

Peter: That’s right. Yeah, I think that’d be great. Because then you know what you’re getting into. And maybe when you get that B in the economics class, you’re not feeling like you failed, that that is a much more typical grade than what you expected.

John: I think a lot of people might object to the notion that there should be the same GPA on the grounds that you’re forcing students into this curve where if you happen to have a class that does really well you might be lowering their grades, which doesn’t always happen in the STEM field, but I think there might be some resistance to that. But the notion of transparency and making people more aware of that early on might help reduce that differential effect by gender.

Peter: That’s right. And I think also making departments aware… the implications that the level of their grading has on things like the gender composition, should matter. I don’t want the way I grade my class to be a deterrent for women taking economics classes. I think that that’s bad. And I’ve changed the way I grade a bit in response to this. If other disciplines are going to give much higher grades, and I want to have a good gender representation in my class, I’m still assigning the same amount of workload. So I’m not making the classes easier, but I am changing the level that I’m curving around, and I typically teach a class that has over 100 students in it.

John: And this is especially important because a large share of the gender differential in earnings is tied to major choice, because the fields that women are more likely to enter tend to offer much lower wages on average, while the STEM fields tend to offer much higher wages. So the differential impact on major choice has quite a bit of an effect on the gender wage gap.

Peter: That’s right. And this is potentially a very low- cost way, at least from a pure financial standpoint, to changing that. It’s not going to solve all of the gender issues in economics or in STEM fields. It’s not going to come close to solving it, but it can put a dent in it, where it’s often very costly to change something like that. Changing a climate is expensive and difficult.

Rebecca: I think I would have been one of those students that would have moved. I’m a person who does a lot of coding and things and could easily have been in computer science. But I’m not, I’m in graphic design, and it’s a field that is much more populated by women. I got much higher grades in the things that I did, and I was very sensitive to grades. So as a student, if it was something that I didn’t do well in, then I thought I couldn’t do well in. And I had a very fixed mindset when I was earlier on in my career. Obviously, it’s much different now. I do much different things and explore other things. But at that time, I think I was incredibly sensitive to that. If you came in as an A student and then it’s like, all of a sudden you’re not getting A’s, it’s like “Woah…”

Peter: Yeah.

Rebecca: “…maybe college isn’t for me, even.”

Peter: Yeah. And that’s just really unfortunate that decisions will get based on nominal differences in these grades, that don’t really have content outside of something that is relative, relative to whatever the professor’s curving around. Another advantage to doing something like this, too, is potentially getting spillovers. Because once you change the gender composition a little bit, then that makes the environment more comfortable. And then that can have multiplying effects.

John: So, why don’t departments want to do this to improve the gender composition of their classes and to get more students in their classes and persisting to graduation in their majors?

Peter: Well, this is sort of where the equilibrium aspect comes in. Because even if you want to do this, everybody ends up ratcheting up their grades, because some departments really need more students. And in order to attract students, you’re going to have to do something if the students don’t have an innate demand for your field. And people have been talking about this for some time, that demand for the humanities has been falling and what to do about it. So, I think that the pressure’s sort of on both sides, even if the STEM departments decided, “Look, we want to have our grades be a bit higher for this reason,” that’s going to lead to response by non-STEM professors as well. So I think it’s hard to get rid of the grading differences across fields, absent doing something that’s pretty draconian.

John: By imposing a grading standard that’s equal across departments.

Peter: Exactly.

Rebecca: I wonder if initiating conversations between the humanities and STEM fields about the interdependence there actually are on those two fields and that people in the humanities should learn things about STEM and people in STEM should learn things from humanities and that there could be more of a relationship there that would actually help start to equalize some of those needs, because there could be some referrals to certain classes or other things that might help boost some of those relationships. I don’t know.

Peter: That’s right. So right now we’re playing sort of a non-cooperative game and move it more towards a cooperative one. Yeah.

Rebecca: Yeah, it’s competitive rather than thinking about “Why are we at a liberal arts institution? And what does an environment like that or general education offer?”

John: Well think about what would happen, though, if grading standards were relaxed in the STEM disciplines. More students would major in them, enrollments would tend to fall in the non-STEM field, and that would tend to lead to fewer jobs for non-STEM faculty. And maybe that’s not something that departments would react well to, and they face the same incentive to just inflate grades.

Rebecca: I don’t think my suggestion was necessarily to do something about the grades directly, but rather to start helping to increase some demand for humanities and helping out some of those…

John: You mean, more interdisciplinary programs?

Rebecca: Yeah, I’ve been thinking a lot about, even for my students, the benefits of them exploring a lot of different disciplines to be more effective in their own, and understanding how other people problem solve, and think through problems, and explore the world.

Peter: I really like that too because if you end up providing these skills that are then rewarded in the marketplace through taking some of the STEM stuff, but combining it with the humanities, that will increase demand. And then that will naturally even things out a bit more.

John: And you could move more from STEM to STEAM.

Rebecca: Hey, I didn’t say it, but I’m totally a STEAM gal. [LAUGHTER]

John: Well I was expecting that at any moment. And just in case some of our listeners don’t know what STEAM is, would you like to…?

Rebecca: Yeah, included the A …it’s for the arts, [LAUGHTER] which I am a part of.

Peter: Yeah, they’re doing that at my kids’ high school, you know, having a STEAM program to try to get that integration happening a bit more.

Rebecca: There’s some research happening about the need for these creative outlets to help with innovation and things in STEM fields. So, there’s certainly some research suggesting this interdisciplinary approach to all of our disciplines could actually improve the work that we do.

Peter: Makes sense.

John: And many STEM students tend to specialize as much as they can in their field. So, that broader training could be useful in the future in making them more creative perhaps.

So, we always end with the question: “What are you doing next?”

Peter: Well, it’s been interesting, nothing quite related to this. I got derailed because I’m actually an expert witness in the Asian American discrimination case against Harvard, and that is taking up all of my time. We released a couple of papers on legacy and athlete preferences at Harvard, and we’re going to be writing some more out of my reports for that case. That’s what’s keeping me busy.

John: Very good.

Rebecca: Well, thank you so much for joining us, this was really interesting, and I hope that it launches more conversations across disciplines.

Peter: Thanks for having me, I really appreciate it.

John: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Savannah Norton.

[MUSIC]

111. The Business of Academic Dishonesty

There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, Dr. Liz Schmitt joins us to discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses. Liz is an economics professor and Acting Chair in the Department of Economics at SUNY Oswego.

Show Notes

Rebecca: There are a number of websites that market themselves as study tools and tutoring services that are used by students as tools for cheating. In this episode, we’ll discuss how these sites work and the steps faculty can take to protect their intellectual property and the academic integrity of their courses.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Our guest today is Dr. Liz Schmitt. Liz is an economics professor and Acting Department Chair in the Department of Economics at SUNY Oswego. Welcome, Liz.

Liz: Thank you.

Rebecca: Today’s teas are:

Liz: Cocoa and coconut.

Rebecca: Interesting.

Liz: It is. It’s my favorite. And I actually nagged John until he bought me more.

Rebecca: Excellent.

John: And I have ginger peach black tea.

Rebecca: I have Lady Grey today.

John: We’ve invited you here to talk a little bit about the growing problem that we’ve seen with the growth of online services that seemed to be designed primarily to facilitate academic dishonesty. I know you’ve had some issues with that in your courses recently. So, we thought this would be a good time to talk about that.

Liz: No, excellent. My post-traumatic stress hasn’t been maximized by the issue. So, let’s talk about it some more. But seriously, actually, it’s a really important issue. And I think our faculty are just not as well informed of this as they need to be. I think in the coming year, it’s going to become my mission to talk about this a lot more.

Rebecca: Can you talk a little bit about the incident that just recently happened to you to give some context and to help faculty understand how these kinds of things play out?

Liz: Certainly. So, really for years, even with the growth of the internet, if you’re using a publisher product… and it used to be sort of paper test banks and end-of-chapter questions… they show up on the internet. Some faculty just aren’t careful and were just posting them on non-password, open web pages. And then sites begin to grow where publisher content was widely published, by students often, with answers and answer keys to sort of pay it forward to future students who might be reusing it. I think what struck me this fall was the extent at which original content (because of this problem I had given up on publisher content)… but my original content in terms of my questions were being uploaded, essentially in real time. And within a few days, custom answers were being made by some of these sites. And this is really a new step in it, because it used to be the easiest way to get around this was to be writing your own content and guarding that content. And we find unfortunately, in September, that that was not enough.

John: The problem isn’t entirely new, though. For decades, maybe for centuries, fraternities and sororities and so forth, have kept files of old exams. But, the internet allows that to scale much more extensively and creates profit opportunities for people who facilitate those services. So could you tell us a little bit about some of the sites that do this?

Liz: Well, let me say this takes a step from the fraternity/sorority test file with this obvious broadening of the audience online, and no longer in geographic proximity. So, social affiliation and institution is no longer an indicator of what’s available to you. It’s searchable. So, it’s fraternity test file on steroids. It’s really one of the issues. But the other issue is these online proliferation of sites correspond to publishers emphasis on their digital content. As publishers really deal with essentially trying to get out of a used textbook market and kind of adapting for new ages of information, they really push their online content… which means faculty are relying more on that online content to operationalize many best practices such as retrieval practice for their students. So, it’s also more useful for students to have access to this. But, I think what really happened this fall is their ability to customize in short frames of time really eliminates the ability for me to just change up assignments in order to control the problem. So, really for low monthly fees, Chegg.com is what I ran into trouble with and for the low, low price of $15 a month they can upload questions. There’s a limit on how many questions, I think, they get a month but students potentially being strategic about it can upload questions and then get customized answers. Caveat emptor, some of the answers were great, some not so great and downright strange, which is really how I gleaned on to the problem to begin with. [LAUGHTER] And there are other sites. Chegg has some competitors I would say Chegg’s probably the market leader in this, but there are some competitors: Course Hero…

John: Course Hero has been out there for quite a while. My first exposure to this was about four or five years ago, when I had a student post in their online class that they were a member of Course Hero and they encouraged other people to join, mentioning that they could get their membership just by contributing a certain amount of graded work from the course to the site in return for that membership, and asking them to use her code so she could also get some credit for them joining. I had a little chat with that student about academic integrity and reminded her that if she posts anything on that site it would have to be taken down and she was going to be reported for this. This type of thing seems to be increasingly more common.

Liz: …much more common. And then you even have an issue like Quizlet. And one of the issues like Quizlet is students actually use it to create flashcards to study. So, I actually think there’s some honest intent for students that often come on to Quizlet. By making quizlets out of your questions and things you ask though, they actually create something for students that come after them. But, often Quizlet is really set up for students to actually create flashcards for different kinds of courses and study. But that’s really the leftover intent. And then, of course, there are online paper mills sites that actually offer existing papers… and again, narrowing topics can get around that and just not say “Write a paper on any broad topic…” that’s kind of really asking for trouble. But again, the customization that has come along makes that even more problematic as well.

Rebecca: And it’s somewhat affordable. It’s not incredibly expensive to be a member of these sites or to get a paper written for for you.

Liz: Yes, Chegg’s membership is $15 a month. So, that’s incredibly affordable. There have been sites… they come and go… kind of on a dark webish sketchy servers… but there are also sites offering to just take your online course. Those tend to be $1200 dollars, and up… Those sites get to be more pricier. But frankly, if you were a faculty member serving an institution with a more affluent student body, you should be very concerned about that.

John: Many of these sites also provide custom paper writing services. So, even if you’ve specified a narrow range of paper topics, students could still order a custom written paper.

Liz: Exactly. I would be fair to Chegg.com… that Chegg com responds within 24 hours in my experience to DMCA requests. So, they’re actually very responsive. And they take them down immediately. And they send warnings to the account that you posted things that you didn’t have permission to post. Chegg.com also cooperated with an honor code request, basically giving me upload dates and the emails of the members that uploaded content as well. So, they will cooperate, like I said, a pretty rapid fashion. I think it really, in the area of legality, it’s not clear if you correct yourself but become this conduit for things to happen, then, I don’t know what kind of legal responsibility is going to come there. It’s sort of beyond my expertise. But, I suspect it would take big players it takes like publisher lawyers and things like that, to really come up against this and demand that you really set up a situation that facilitates the repeated stealing of our content. Because it can be an intellectual property whack-a-mole game, because they take it down, but then another student post It from entirely different account and Chegg and these sites are kind of saying they’re not responsible for that. Legally, I’m not sure if you facilitate that, how responsible you are. There’s also paraphrasing tools. Paraphrasing tools… that’s very caveat emptor or buyer beware type of situation… and I actually have an example to show you what can happen when plagiarism tools go bad. Friends don’t let friends use plagiarism tools or this is going to happen. So, let me show you the answer from Chegg expert which was a very nice answer… is as follows: “Expectation of rise in inflation will lead to a drop in the prices of Treasury bonds. This is because the required yield by investors will increase in order to generate higher returns after beating the inflationary pressure.” Great answer, run it through a paraphraser and this is what you’re going to get: “Desire for ascend in the expansion will prompt drop in the cost of the Treasury bond. This is on the grounds that the required yield by the financial specialist will increment so as to produce higher returns subsequent to beating the inflationary wait.” [LAUGHTER] So, that’s not a thing… that is not a thing. I think the odd choice of vocabulary… Here’s words that college students almost never used. And here’s 10 of them right now, in the same paragraph. Obviously, that’s the warning sign. And the idea is, of course, context is important. Certain words that are used by convention or tradition in financial markets. Something that’s really a synonym, technically, in the English language is not in the context of financial markets.

John: A traditional way of catching students who were doing this often was that they were providing you with something that seemed a lot stronger than their other work.

Liz: Yes.

John: And this sort of reduces that down to where, perhaps, it may not be as obvious in all cases that someone is doing that because the quality of the work will no longer look exceptional in the same way.

Liz: Yeah, the exceptional quality of work, you’re right… because the first answer actually does set up my radar because it uses a complexity in sentence structure that I might not expect in this answer. But, I think the other one was just immediately obvious because the word choice just doesn’t fit. So, even if you were actually looking around on Investopedia, or some other website to borrow language, it’s not language that would be used by someone writing for that.

Rebecca: Even as a non-expert in the field, I could tell you that that is not language of the field.

Liz: “Desire for ascend in the expansion?” Yeah.

Rebecca: No, no.

Liz: I think pretty much called that one out.

John: But if you have some foreign students in your class, that sort of thing might sometimes happen. I’ve certainly seen examples where students have used synonyms inappropriately. So, some of this may get by.

Rebecca: In some cases, though, if it was a student where English wasn’t the first language, you would probably have some assignments and know already the kinds of mistakes that that student would make.

John: Yes.

Rebecca: So, you would have an idea of whether or not that would be consistent for that student or not.

John: We touched on the issue of hiring other people to do a student’s work, but could you talk a bit more about how the gig economy may play a role in academic integrity issues?

Liz: Well, the gig economy is basically the idea of you give me your specialized topic and I will write a paper custom for you: words, APA or MLA, the formatting etc., ready to go… Course completion, I will take your online course for you… paper writing services, again, is sort of that issue. And again, in ome of these paper writing services, it’s sent as a Word document, and they don’t actually strip the properties of who created that document.

John: And one of the things that has helped facilitate this, though, is the international reach of the internet. In countries such as India, where incomes and wages are a lot lower, you can pay people relatively small amounts to get reasonably high quality papers written in very strong, solid English

Liz: Right, exactly.

Rebecca: So, what can faculty do? This sounds very dismal.

Liz: It does sound dismal and there’s no solution that doesn’t involve some time on task here. And I think that’s a big problem. Because most of my faculty colleagues, I don’t really know anyone who sit around doing nothing… who has extra time to deal with this, and that’s got to give. Scaffolding is a common way to deal with this. So, it’s a recipe for disaster to assign a paper, be very unspecific about the topic: “Do any topic in post-World War II US history out…” That’s going to be bad. You want to focus the topic, then you actually scaffold. When do you want a thesis statement? When do you want an outline? an annotated bibliography? So, trying to have the pieces turned in really prevents their ability from coming out to get a paper or it’s really the red flag when a student several weeks into the process suddenly wants to change topic, having had struggled earlier with the topics and then they tell you when they want to change, you need to get ready for what that paper is going to look like. And you can scaffold in other ways, even on homeworks: you reference. I reference “Using the model from chapter seven,” “using the model we talked about in class in week six” or “using the discussion issues that were brought up in discussion three.” And so you can’t easily post those and get answers because then the students would have to provide the context, which defeats the purpose as well, of the questions. Algorithmic questions, and algorithmic questions can be posted and solved, but algorithmic questions are really after the fact, it makes it a lot easier to snag people. Because, if you just have a general problem…. So, I had a problem on the internal rate of return, which is a common time value of money concept in the field of finance. And while I think students really got their answers from Chegg, unless they were kind of lazy enough to exactly copy the wording of the answers, I couldn’t really get them because there really was a way that you solved this in the spreadsheet. So, the solution should look the same. So, you get around that by algorithmic questions where the numbers are unique, and then you know, exactly who uploaded that question. And in a learning management system, it’s easy enough to regenerate numbers every semester or year that you’re teaching the course. So, there’s a new set of unique numbers. So, that’s more about enforcement, I think. Questions that work from very specific data, maybe a data set, or quotations. So you can start with: “Mitt Romney, in a debate in 2012, called China a currency manipulator. What does that mean?” And so that quotation again, makes it harder to find general answers to that question. Questions that reference their own experiences, where they have to call up a specific experience themselves, and expound on it, and apply it in class. And current events. So, if you do actually have a very current event, then you can actually prevent going back in time and trying to find older questions, because it’s a current event as well. And so you’re constantly changing. So, you might be testing on the same topic, but you’re constantly changing the context. And your question has to be written so that you forced them to address the context as well.

John: I’ll often ask students to find an example of something in the last six months that illustrates some concept that we’ve just discussed in class. Because if I only offer the class once a year, they’re not going to be able to go back and find earlier examples from the class.

Liz: Right. And then I guess my least compromised questions are ones where they actually create a graph using the Federal Reserve Economic Database based on macro data or financial data. And again, that’s right, because it has to be current. And from year to year you change the time period, or what you want them to look at, or different measures of inflation, things like that. And then you can really grab that, and I have yet to see a FRED question appear on Chegg.

John:(…and FRED is an acronym for Federal Reserve Economic Data.).

Rebecca: What is your process for monitoring because clearly, you’re doing some monitoring.

Liz: Right. Well, when I found the extent of this problem in September, I basically went forward several weeks, and just copied and pasted questions into the internet search engines. I also ended up at least getting a subscription to Chegg so I could look, because the other issue is if someone screenshots an image of a question from their learning management system that might not show up in a Google search, but it’s going to show up on Chegg because Chegg actually has the text… kind of an alt text… that comes up, and that will show up as well. And so I started doing that. I opened a Google doc. And then every time I found one, I knew I had to change the questions in my graded activities. And then I would put it in the Google document. And I ended up with a 20-page DMCA document to Chegg about all of the questions I wanted to take down that take me through about week 10 my course. So… to be continued… as to how many I will find. But really, the monitoring really comes up at first when you get just a very unusual answer, or you start getting all the same answers. So, in a class of about 37, there’s about 10 students that were trying to tell me the same thing, especially if they illustrate a concept… I don’t ask for a numerical example… But they give me one. When you see the first person do that, you’re like “That’s great.” And then you think, “Well, why would the second, third and fourth person choose the exact same numbers to illustrate this concept?” There’s only one reason and it’s not a good one.

John: You mentioned the paraphrasing tools making it a little harder to find when people were trying to copy materials. There’s other tools out there too, though, which are online plagiarism detection sites where students can get a paper that they think fits the requirements of the assignment, upload it to that, see if it’s found, and if it is, just make minor changes and resubmit it until it ends up with a relatively low plagiarism score. And complicating this a bit, is that some of the major paper mills advertise that they use a TurnItIn service to check the papers that they sell for plagiarized content. Because one of the issues for the companies that are selling papers to students is that sometimes the people they hire to write papers simply plagiarize existing material. So, this makes it a little bit harder perhaps to detect that sort of work for hire. What might you do to detect papers that have already been run through the TurnItIn or similar systems and modified to come up with low plagiarism scores?

Liz: Well, I think you have to go backwards from that and recognize the limitations of those tools. I was never a big fan of TurnItIn. SafeAssign is just kind of a starting point. SafeAssign often gives a lot of false positives as well. Because if people are citing sometimes similar sources, which you would expect, then you’re going to see that pop up in SafeAssign. So, I think you got to go back and say, are you choosing the topic that is sort of paper mill proof, or things like that? Downloading when you have electronic submissions, which I do, and downloading them can also pick up issues of extra characters designed to trip it up. So, people put characters and they change the color to white to try and trip up that detector and that easily shows up when you highlight them. Those are some of the things that I would use. wWe haven’t had TurnItIn on our campus. We had SafeAssign, which I already knew the limitations, but the tell is you read it, and you’re like I’ve met five college students that write like that… in 23 years. So, I’m now deeply suspicious of that. Often the language doesn’t match, which is why scaffolding also becomes important… because the language doesn’t match what’s been done earlier in terms of: if you don’t think the student wrote the paper and you can’t find the source, you invite them in. And you ask them about where they found the source. Or I really love what you said about the impact of inflation on average households in the 1970s. Could you explain how you developed that? …and they won’t be able to and as a result, they’ll often say, “Well, I wrote this paper and I immediately forgot everything.” And faculty members get caught up into this. “Well, I have to prove a negative or I have to prove that that’s not possible.” And you don’t, I think you’d say: “On your face, that’s a really ridiculous argument. Either you need an MRI right now, because something wrong has happened. You’ve had a stroke and you need help, or you’re not being honest about where that paper came from. And you’re allowed a chance to defend yourself from these charges. But that defense isn’t reasonable. And I’m going to move it on.”

John: Pretty much every time I’ve run across that and brought the students in, they almost always confessed pretty quickly when they realize that they can’t explain what they had written.

Liz: Yeah, I think turn it in has these separate issues too. Is it ethical? We’re complaining about our intellectual property being posted. But, students that do write original papers, that’s their intellectual property. And we’re forcing them by virtue of getting their degree and meeting requirements of a course, to release some of that somewhat. So, I think there are issues of privacy and property, ethical issues that I know a lot of other faculty out there have just really pushed back against for years with turn it in. Others argue it’s about us policing students by requiring this prevention method, we’re almost taking the assumption that something’s going to go wrong, that it can create a hostility and an adversarial relationship that we don’t want in students. I think there’s some truth to that as well, which is why it becomes so important to think about the design of your course and your learning activities, because it’s so much better to prevent this and to make it very difficult than to deal with it afterwards. As I tell my students in my annual first day of class, don’t test me on this because I fail people at least once a year for this kind of thing. I actually say this is the worst part of my job, and I hate it, but I do it. So, that should tell you how important it is to me.

Rebecca: I think that it’s sometimes it’s a little more obvious to faculty how to prevent some of these things from happening in an face-to-face class, because you can be doing in class assignments, you can have them working on things in class and see their progress. But that doesn’t play out maybe as easily online because you don’t necessarily know what they’re doing. So, do you have some strategies of how you scaffold or do things a little bit differently online?

Liz: I would back up again, because some people say, “Well, this is why I don’t teach online, because I’m worried about ringers and things like that.” But, often people that tell me that do assign papers, and I say “Basically, anything that’s done outside of class is susceptible to these sites…anything.” And so really, again, there’s a lot of face-to-face classes, particularly in STEM that are using homework packages that students complete on their own. And then there are plenty of classes that assign papers that are done outside of class. The idea is, is that the in-class possibility of a face-t- face class does provide this check. But in an online class, you really compel them to actually talk about things in a discussion. And so you can use your online discussions to lay the groundwork of what they want to talk about, requiring them to reference discussions within the course as well. And so that’s one way to try and mimic that check that in-class does. And there’s proctoring software, which again, is potentially foolable as well, but there’s also design. So in an online class, the idea is no one thing should be worth a lot. Any one thing should be worth less than 20% in my opinion, because the idea here is if you want a ringer, you don’t want that ringer, by just taking a single exam, to move your letter grade significantly. You’d need a ringer for the whole course or a really, really good friend to do this for you. So, that’s another thing and in designing them exams, you can time them, not necessarily just time them but there’s one question at a time, no backtracking options. So, you want to think about a very structured way to require students to demonstrate understanding that just make it a lot more difficult to outsource.

22:08 redo

John: You can do the same type of scaffolding online as you do in a face-to-face class. You just have different stages, as you said. In the first stage, you could have them submit a thesis statement. In the next stage, you could ask the students to submit a bibliography, followed by an annotated bibliography, and then a rough draft and a final draft of the project . And that’s not really much different than it would be in a face-to-face class.

Liz: And all in the same place are all of these writing samples because that’s how you’re communicating with students: via emails, course messaging, discussion forums, and then other graded work. So, in some sense, there’s a large body of written work to form a basis for your suspicions or concerns.

Rebecca: I’ll also add that we focused on lot on written work, test questions, and things like this. But, the same kind of plagiarism can happen with images, it can happen with code. I’ve had those same experiences in my classes as well…. sharing of digital files to make a particular design and boy does that look kind of similar to something I’ve seen before or an image that’s being claimed as their own or not documented where it came from. So those same things happen. They don’t play out in the same way in Chegg and some of these other sites, but those same practices happen through like a gig economy or just sharing amongst other students and when their digital files are a lot easier to share them when they were physical things.

Liz: Also in creative fields, there’s something worse in the way that’s sort of accepted, because if you look at fashion design, the ripping off of top designer early designs to then the knockoffs is astonishing. There’s a photo that has a Manolo Blahnik sandal, frankly next to an Ivanka Trump sandal, and they’re the identical red sandal. And it’s really just you slapped a different name on it and use different materials, perhaps. And obviously, they’re about six months apart in terms of product cycle. So, I think people in creative fields see that recycling, even in the music field, whether they get permission, you see music, that’s a remix of other older musicians, and you need permission for that, but you don’t necessarily know that when you’re listening to it and enjoying it. So, I think you actually see in some of the real-world situations how visual borrowing and frankly, stealing, is kind of accepted in some of these fields by some very successful people. Whereas, in the written world, when authors get found with plagiarism, it’s considered a big deal. And it’s kind of very embarrassing for them to get caught. Books can be recalled… things like that. It seems like written word… there seems to be more of a consequence, for now, when that happens, whereas in creative fields, that’s not always true.

Rebecca: There’s a lot of visual and audio copyright cases, though. And that’s where that tends to play out. The fashion example that you’re giving… those are considered functional things and not considered creative. So therefore, they’re not protected by the law in the same way.

Liz: Exactly. But, I think for what students observe, and behavior that they might emulate, when they look at professionals and their choices, that distinction isn’t going to come up.

Rebecca: One thing that I always argue is that when we talk about intellectual integrity, that copyright comes into the discussion in particular fields, because it is sometimes a common practice. And it’s there for a reason. we iterate on our culture, our culture creates new culture. It’s not a crazy concept, but you need to know where it comes from. And you need to provide attribution and in an appropriate way, which is no different than following MLA or APA or some other thing. So, I think that’s always something that people should be thinking of, and that when you’re having written papers, and there’s images and things in it, that you’re also thinking about that part of the content as well and not just the written word.

John: One other thing that I’d like to bring up is an earlier podcast that we did with Judith Boettcher, who talked about one way of avoiding this issue by having group projects that provide students with a lot more autonomy, but in a very structured fashion. And that’s perhaps a way of getting around this where students can take more ownership of the project and they create more of the project as a group, which would make it a little harder to engage in academic dishonesty. And we’ve also, in previous podcasts, talked about some open pedagogy projects, where the work that students do is posted publicly. And if they know that when their work is posted publicly, and they’re copying something from anyone else, it’s much more likely that they would be found and they’d get in some trouble later. So, those are two other things, perhaps, that might be ways of reducing the incentives for academic dishonesty.

Rebecca: Community based learning is another one… or service learning where you’re doing projects with the local community, because all of that context would be unique every time you’re doing something. So, that’s another opportunity for grading those assignments that really aren’t reproducible, and would be really, really hard to get an answer for… unless the person lived in the same community.

Liz: I agree. I think the biggest challenge is in some of these courses that are tool courses. In some courses you’re acquiring the tools that you would learn for projects and to consult, etc. And so when you teach these tool courses, it’s not always appropriate to have these kind of finished product things, because they’re in progress of assembling that toolbox that they’re going to use. And this is where reliance on sites like Chegg become a big problem.

Rebecca: At the beginning, we started with the idea that faculty aren’t always aware of these tools, or even the ways that students try to manipulate the system. Can you talk about ways that we can help increase this awareness with our colleagues?

Liz: I was just talking about this this morning with another colleague and we were bemoaning our Chegg purgatory this semester, and she says, “I just don’t think other faculty realizes… or how can we be the only ones that care about this.” So, I think some of them are honestly unaware because faculty aren’t always in sort of the student space and understand what those crazy kids are doing these days. So, I think, in some sense, faculty that have been more tuned in to creating learning community or kind of developing a relationship with their students are more likely to get ideas of what’s going on as well. But I also think there’s some willful ignorance here… this whole “Well, I didn’t know.” But I took steps to make sure I didn’t know because once you do know, it obligates you to do something, if you “don’t know,” and I’m using quotes, which is not helpful on a podcast, air quotes, but if you really don’t know that you’re MyLab component of the course is completely compromised, that doesn’t obligate you to think about changing up your course, and the weighting of activities and what activities. Once you know, it really obligates you to act as a faculty member. And so some people say, “Well, I wonder if my stuff is up there?” and like, “Oh, it is.” I don’t even have to look, I can say “Yes, it is.” And then academic dishonesty takes time. I’m really at 40 hours and counting with academic dishonesty documentation, DMCA documentation, and then reworking items in the course to deal with this issue so it wasn’t a lost semester. And I’m a full professor, think about an assistant professor not only trying to balance and develop their research agenda in conjunction with this, but also not wanting to rock the boat with unhappy students. I’m going to be getting a nice bottle of something sparkly, when I read my faculty evaluations in January. They’re going to be lit. Perhaps we’ll do a dramatic reading and have a tea party with tea and maybe something stronger. But, you know what? I can weather that. I can take that. And that’s going to be an issue. And frankly, I’m really proud of this institution, about how administrators really back faculty, enforcing the integrity of coursework and the degrees. And I know that doesn’t happen at other institutions, frankly other institutions that are maybe more tuition dependent and driven… that are unwilling to make steps that make students leave with financial implications. So, in some sense, this is really one of the best environments here at Oswego to actually try to enforce these policies as well. So, I think that’s one of the reasons that faculty don’t act, because there absolutely is some blowback to that.

John: And faculty might also see that each year they teach the same course, their students are doing better and they might be very content just to see that improvement in the scores so that all the work being submitted looks more like the best work from the year before. And it’s really easy to passively accept that.

Liz: Well, I would actually note that what’s more astonishing, is not the extent that Chegg has corrupted these issues, but there is a significant contingent of students that do not use it. Maybe they are unaware of it. Maybe they decide they don’t want to use it. But, when someone says, “Well, I can’t believe how many students cheat,” I would say, “Well, I can’t believe how many don’t, given the incentive structure.” And so that’s somewhat encouraging. But, also I had to decide if I was going to email the entire class and say, “Look, I’ve seen this happen. And you have to know that this is not allowed, that I’ve already made it clear in the syllabus. This is not allowed, and this is what will happen.” And it was a real-life decision about the Streisand effect, because you have to wonder if 20% of the class is saying “Wait, there’s a Chegg and we can get answers there? Alright!” You didn’t know if you wanted to clue them in, but I figured I had to act under full information.

Rebecca: I think it’s also really interesting when you start seeing students wanting to apply for jobs at those kinds of organizations, and then what kinds of conversations you’re going to have with those students, when they say, “Hey, I found this really awesome job that I want to apply for at Chegg.” It’s like, “Well, think about that. What does that mean? And what are those implications? And where are the ethics behind that?”

Liz: Well John, and I were talking earlier, there are Faculty Fellows at Course Hero, and some of them have made a name about teaching in the profession. John was probably more polite about that. One of them’s at a conference and I think I would stand up and say, “How do you reconcile partnering with a site that facilitates academic dishonesty and intellectual property theft every single day?” I would be curious as to what the answer is.

Rebecca: The pay is really, really good? [LAUGHTER] I don’t know.

Liz: I guess. The one I’m thinking of has a pretty sweet gig right now, so I’m not really sure.

John: this presentation was right after one that was riddled with references to learning styles, so I saved my powder for that one.

Rebecca: So we always wrap up by asking what’s next?

Liz: First, I’m already kind of brought this up in faculty assembly. And I’ve brought it up in academic meetings, because I’m acting chair this year. So I brought it up in leadership meetings. And I hope we can actually do a workshop through our Center for Excellence in Learning and Teaching. So, maybe a breakout workshop talking about our experience, and try and really broaden the understanding of the issues here. I’m also working with our faculty union, the United University Professors is also very interested in compiling violations of intellectual property rights, and trying to deal with that and push back against those sites. And I’m actually sharing my DMCA documents. So I made an editable form for some of the big sites, so you can easily go in and change them. I made a template for what you would ask the Associate Dean to fill out in order to ask for upload information from these sites as well. So trying to minimize the work involved for people who want to do this and take action. And then, finally, I’m just looking ahead about how I’m going to really redesign this course that I’ll teach a year from now, and to motivate and enforce original work.

John: And I should note, we’ve also been offering workshops for at least seven or eight years now.

Liz: You’ve been offering workshops on these sites and things like that….

John: …and attendance has been generally limited. We’re lucky if we get 15 people.

Liz: But you don’t have me. I’m a draw. I’m a draw… a star. [LAUGHTER]

Rebecca: …especially if we could do the dramatic reading of the examples.

Liz: Exactly. There will be a dramatic reading, and there will be sufficient supplies of snark. And I think it’ll work. But, I actually think maybe a case study. So, we come and we talk about these sites, but I can sit down with other faculty who’ve had this problem. And this is what I found… this is how I found out about it. And this is what I did about it, and this is what you should be doing about it.

Rebecca: This has been super informative. Thank you very much, Liz.

John: Thank you, Liz.

Liz: Well, thank you for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.

107. Project NExT

Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, for Assistant Professors from the Math Department at SUNY-Oswego join us to discuss how our math department is transforming its instructional practices through the use of professional development opportunities provided by the Mathematical Association of America.

Show Notes

Transcript

John: Faculty beginning their teaching careers often rely on the teaching methods that were inflicted on them when they were students. These practices are not always consistent with evidence on how we learn. In this episode, we examine how one department is transforming its instructional practices through the use of professional development opportunities provided by its national professional organization.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today we are joined by four assistant professors from the Department of Mathematics at SUNY Oswego. Our guests are:

Sarah: Sarah Hanusch.

Rasika: Rasika Churchill.

Jessalyn: Jessalyn Bolkema.

Zoe: And I’m Zoe Misiewicz.

Rebecca: Welcome everyone!

John: Our teas today are:

Rasika: I’m having Earl Grey.

Jessalyn: I just poured myself a cup of lemon ginger.

Sarah: I’m not having any tea today. I’m not much of a tea drinker.

Zoe: I’m not having any tea today either. I just haven’t unpacked to that point yet.

Rebecca: And I have… English afternoon.

John: And I have Bing Cherry Black Tea. So, we invited you here to talk about Project NExT, which is something that people in our math department have been involved with. Could you tell us what Project NExT is?

Sarah: So, Project NExT stands for New Experiences in Teaching. It’s a program that is sponsored by the Math Association of America that brings new mathematics faculty…so you have to be in your first or second year of a full-time job…but they bring these new mathematicians in from all over the country to teach them about active learning.

Rebecca: How did your involvement, or the department’s involvement, with Project NExT get started?

Sarah: I learned about it as a graduate student, and was highly encouraged by a lot of people to apply. And so I kind of brought it into the department by saying, “Dear Department Chair, will you pay for this?” And since then, in part because of my starting it, we’ve encouraged everyone we’ve hired to apply. And as a result, there’s now five members of the department that have either completed or are still in Project NExT.

Jessalyn: Yeah, I will echo that experience. It was something that I was aware of as a graduate student, in part because some of my mentors had gone through Project NExT…it’s now 25 years old…just celebrated 25 years. And so for me, it was something that I knew I was interested in. And in fact, when I visited Oswego for a campus interview, and the department said “Oh, yeah, we have Project NExT fellows on the faculty, and we would be happy to support you in that,” that was a really exciting and encouraging thing about the department.

Rasika: For me, actually, I didn’t heard about that before. But, when I got the job offer, it came with that. I said “Yeah, sure.”

Zoe: I was just hired this past year and so I’m doing Project NExT, but I think I can already see the effects that it has had. It was a program I already knew about, I really wanted to participate in. So, as I was going through the hiring process, one of the first things I would ask the chair at a place was “Would you support an application for Project NExT?” …because it does require a bit of funding. And so seeing that there were already multiple Project NExT fellows in this department was also a good sign for the department as a whole when I was thinking of what sort of department I’d want to be at. And so I think it’s just showing that it’s already been recruiting people who are interested in it already, at this point.

Sarah: I was just going to clarify a little bit about how the funding for it works. There’s actually no fee to participate in Project NExT. The way it’s organized is that you attend special sessions at three of the national conferences in mathematics. So, you attend two math fests in the summer, and then the joint math meetings, which is in January. And so these are big nationwide meetings in mathematics. And so the idea is that you’re going for some special sessions during the meetings. And then your first year, you go for a couple days pre-conference for the really heavy duty workshop. So, the financial commitment from a department is just the funding to go to those three conferences.

Rebecca: You mentioned active learning. Can you talk a little bit more about how those workshops and things are structured?

Zoe: There were a lot of workshops about active learning and just using evidence-based pedagogy, so saying not only active learning is good, but we have evidence to support it and here are some of the things that you could do in terms of active learning. And all the sessions obviously are structured with that in mind. So, we’re not just sitting there listening passively to someone tell us about active learning, but they really make sure you’re doing something, whether it’s a fun little game like building a marshmallow tower, or some other interactive activity in each session. The sessions aren’t only about active learning, there’s a lot about inclusivity and diversifying the profession. So, a lot of sessions on that, or maybe I just chose sessions on that. But, there’s also a whole professional development stream. So, there’s stuff about how to get started in your career in terms of grants and so on. It’s really a lot of everything in there.

Rasika: It’s categorized like if you interest on the tactile learning, so are you interest on the group work, are you interest on some other…you know, inquiry based and mastery grading and so forth. So, depending on your interest, actually, they give more opportunity to listen, go talk with people and have a conversation: what they had, what they tried and what failed and what succeed. Which is like a really nice thing for us, as a beginner, to see what people have gone through and what I should expect, and so forth. Actually, I was interested about the whole program.

Sarah: So, they do some three-hour breakout workshops where you get to go based on what your interests are. So, I did one that was focused on teaching future educators because that’s my background, but I doubt any of these other ladies chose that same session because that’s not their expertise and not what their job is going to be about fundamentally.

Jessalyn: I will add, I attended two workshops that stand out to me in retrospect. One on making active learning intentionally inclusive. That was all about inclusive pedagogies and ways to incorporate group work in the classroom in a way that benefits all students and allows all students to participate fully. I also did a longer breakout workshop that was building a toolkit for student-centered assessment, that was all about learning objectives and exam structures from a more experienced instructor. And then there are also facets of Project NExT that extend well beyond the physically meeting in person. So, as Rasika mentioned, there are lots of ways that you can navigate the workshop according to themes that are of particular interest to you. So, if tactile learning or kinetic activities are of interest, or you’re really focused on educating future teachers or whatever that might be, you might be encouraged to declare a goal for yourself in your first year related to one of those areas of interest. And then we’ve got little email exchanges that go on for people who’ve declared interest in one of those goals like “this email list is all about mastery based grading, check in when you’ve tried something. check in with your questions.” So, there’s a little bit of accountability built into that structure that these people know what you’re trying to do, and they’re going to check in with you on it. But, then just the larger structure of email lists is that you have this cohort of other new instructors who will fire off questions like, “Oh, I’m teaching this class next semester I’ve never taught before, what textbook might I use?” or “I had this really strange interaction in my classroom, and I’m not sure how to handle it” or “I think this part of my syllabus is just crashing and burning. Help! Has anyone been here before?” And so you have this sort of communal resource and the community experience of brainstorming and problem solving together.

Sarah: …and included in that they assign each of us a mentor. So, a more experienced instructor that’s a mentor is assigned to each person in the program currently, and it’s always someone that is outside of your department. In fact, they will not allow anyone to be a mentor who has a fellow in their department. So, as long as we keep having fellows, we won’t have any mentors here. But, what’s nice is when you do send emails out on that list of “I’m trying this and it’s not going well, help!” you do get responses from your peers. You also see responses from all of the mentors for that cohort, which I think is also valuable because sometimes they have a little more experience than your actual cohort.

Rasika: We have a group that people who are interested on the inquiry based or tactile work, they have their own little Zoom conversation whenever they have time together. You get to know all different schools, what they’re doing and, you know, share your experience.

Rebecca: Would you all like to talk a little bit about how Project NExT has influenced your own teaching?

Rasika: For me actually, I was really interest on the tactile experience from this Project NExT. So, I decided to do some activities this semester starting as a beginner and also some group work. And also something that… not exactly what I’m getting from the Project NExT, but it’s like I will say, part of the SUNY Oswego Reading Group, that I was so interested on the book that we are reading. And I decided to give a couple of pages for the students every week to read, and I assigned them 5% for the final grade that they have to read and write half a page to one-page report for me and tell me what they think. Do they think like it’s feasible for them to change and try and do the things in that nature? So far, it’s really going well, and I have good comments from students saying that “you are opening up different ways of thinking…that we were stuck and never complaining about everything. But, we are now having, you know, in a broader way of looking at the things about growth mindset and so forth.” So, I was speaking here and there like chapters from some interesting books. So, that’s what my experience so far this semester, as a beginner.

Sarah: I think for me, it just gave me a lot more lesson plans and ideas to draw from. I already had a pretty active approach to my teaching, but it just opened a broader view of what kinds of things could work well. Especially in some of the more tactile things available that can be helpful for helping students to learn.

Jessalyn: Within my own teaching, I think it’s been really easy or natural to draw on resources from Project NExT in setting up my class or setting up lessons. When I taught Calc I, on day one, we made zip lines out of ribbon and key chains and measured average velocities and it was fun and it was memorable and it got students working in groups and they reported at the end of the semester. “Hey, remember when we did zipline? That was fun!” and I 100% would not have pushed myself to do something that involved or non standard, I’ll say, without thecontext of Project NExT saying “Oh, just try one new thing each semester.” I completely overhauled the Calc II class to be entirely mastery based grading in response to some of my own frustrations with how I had been setting up my class. And Project NExT supplied a whole lot of resources, a whole lot of people, a whole lot of information and motivation to try something like that, which I think was helpful. As far as department culture goes, I think the fact that we’ve had this many Project NExT fellows and continue to have Project NExT fellows gives us a shared language to talk about teaching. Some shared frame of reference on “Oh, yeah, you know, this person who tried this technique,” or “Have you heard anything about…. “”…Oh, hey, this came through on my Project NExT list.” That I think has encouraged just our conversations about teaching and being intentional in how we’re structuring our classes, or how we’re handling things.

Sarah: I’m experimenting with mastery based grading this semester because of the information you and John got, from your experience in Project NExT. And so your experiments with it last year has led me to experiment with it this year. So, it definitely has changed just how we even hear about new things to try.

Jessalyn: That’s delightful. I appreciate that it’s trickling around.

Sarah: It is trickling for sure.

Zoe: So, I’d say it’s still obviously fairly early. We’re only one month into my first semester after going through the first part of Project NExT. But, I’d say a lot of it has been both an affirmation of things that I have been doing and also it’s sort of given me the confidence to do the things that I was doing even more fully and to advocate for these approaches, even though I am brand new in this department. So, I’m not afraid to send to the whole department email list like “we need to be more positive toward our students and not say that it’s all their fault if they’re struggling. we need to take responsibility for that.” Or just to try things that may or may not work well. For example, I’m doing mastery-based grading just of the homework in my general education math course. And I’m using an online system that,it turns out, is not that great for mastery-based grading of that course, even though I’ve used it for other courses. Students, I think, still benefit from it, but it’s not quite as effective as I might have hoped. But, I’m just willing to try these things and willing to speak up about things, so those are the main impacts in my courses.

John: Could you tell us a little bit about how you’ve implemented mastery learning technique?

Sarah: I think we’ve all done it a little different. Why don’t you start, Zoe, since you were just talking about it.

Zoe: I’ve done it only in the homework, so not in their exams. So, the homework is done online, it’s 15% of their grade. And so for each little subtopic, they have to do a little quiz. It’s five questions: three medium, one easy, one hard, and they need to get at least 90% on it. And they can try as many times as they want, but they do have to keep trying. And so, in courses like college algebra…is the one that’s most similar to where I’ve done it before…the material all builds on itself and it divides nicely into little component and there, I’d say it’s going well. The students complain about it at the beginning, but already after I asked them to reflect on their first test performance, a lot of people said, “Oh, it’s actually really helpful that I had to go back and keep learning these things until I fully understood them.” Whereas in the first couple of weeks, there’s always a bit of pushback about “Why do we need to get 90% on this. It’s too hard to get 90%…couldn’t it be lower?” And then once the results come in, they see it’s worthwhile. The other course I’m doing is similar, the gen ed math course…it’s also their online homework…15% of their grade, but that textbook just doesn’t break down the material into as nice sections and the questions are longer and the grading of the online system is pickier. So, that one has some issues, but the same basic idea.

John: Are you using publisher provided questions then, and tools?

Zoe: Yeah, publisher provided questions and tools.

John: Are you allowing unlimited attempts or is a limit on the number of attempts?

Zoe: Yeah, unlimited attempts, and flexible deadlines too. So, I do say they need to achieve a certain amount before each of the test. But, the idea is that if you haven’t yet mastered something, you can still go back and do it several weeks later. As you keep practicing the material, we keep building on it. So, it’s not that you have just one chance and you’re done. The goal is to get them all to understand it fully by the end of the semester.

Jessalyn: My approach to mastery based grading in my first implementation was to go totally off the deep end, and just structure the whole class with a mastery-based grading scheme. So, what this meant was that I did away with midterm exams, everything was broken down into learning objectives roughly correlated to the sections that we were intending to cover in the textbook. And the primary mode of assessment was quizzes. So, my students had quizzes that they could retake as many times as they needed to. And each quiz had three questions and I wrote problem banks of many many questions for each quiz. And in order to earn an A at the end of the semester, the expectation was something like, “Oh, you need 18 of your quizzes to be three out of three and the rest of you two out of three.” So, it was not a points accumulation scheme, it was just quizzes and repeated quizzes. They also had online homework through web work and that was unlimited attempts. There were deadlines, and they just needed to… there was sort of a threshold percentage associated to an A or a B, or a C. And then I had a few more other activities and elements going on. But, primarily, the structure involves these mastery quizzes. And I owe a great deal in the structure of this class to Laura Taalman from James Madison University, who shared a lot about how she structured her class that way and so I sort of borrowed and adapted from her setup for my experiment.

Sarah: So, my class is pretty similar to Jess’s. The main difference is I’m doing it in a proof-based course, so it’s fewer questions. She had three questions per objective. I have one, because they’re a little bit longer questions. The only exam in my class this semester is the final and that’s only because I’m required to have some common questions on a final exam. So, I had to have a final exam…instead I’m doing weekly quizzes. Each week, we add one to two new objectives. There’s about 20 for the entire semester. So, our first week we had two questions on the quiz. The second week, we had four questions on the quiz, but questions one and two were the same objective as one and two from the first quiz. So, the questions are just going to grow cumulatively…so our last quiz will have about 20 questions on it. Although I did tell them once everyone has mastered a question, it’s just going to say mastered, it’s going to be no new question writing and at some point, I’m going to recycle some of the early ones.

John: Your building in some interleaved practice and spaced practice as well.

Sarah: But, the idea is that once they have mastered a question, they no longer have to do it again. They’ll have the questions for practicing and for getting ready for the final. In addition to these mastery quizzes, I’m having them write a portfolio, which is going to have a little bit more of that interleaving practice and making sure that at the end of the semester, they still remember how to write some of these early proofs and it’s also to focus on the writing aspect. So, to help make sure they’re really using the language precisely. Sometimes with a quiz when it’s timed, you’re a little more flexible, but I want to make sure that they have that precision of language down by the end of the semester. So, I’m sort of balancing those two aspects of it that way. They have “unlimited attempts” in air quotes…restricted by what? …there’s 12 times I could quiz during the semester…13 for something…So, restricted to…they need to do number one all semester long. They can have all semester to do it, but we are eventually going to run out of time.

Rasika: So, for me, I haven’t tried to mastery based grading yet. Maybe in the future.

John: Are there any other new techniques any of you have used in your classes?

Sarah: I’ve done a lot of experiments with this idea of embodied cognition, where you actually have students sort of using their bodies to experience things mathematically. One way that we did this with my pre-service elementary school teachers, I give them a bunch of clothesline, and I have them make a circle. So, you may think, “Okay, no big deal.” But, what happens is, it’s not good enough until it’s a perfect circle. Part of this is to elicit the definition of a circle, because to non-mathematicians, I’m going to pick on you for just a moment, Rebecca, how would you define a circle?

Rebecca: One continuous line that’s in a loop.

Sarah: So, a lot of times they come up with something like that. Well, how does that distinguish, though, a circle from an oval. So, it’s not really a precise definition of a circle, right? With the precise definition is being it’s all of the points that are a fixed distance from the center. But, what happens is, by forcing them to make their circle better and better and better and better, they actually all know that’s the definition of the circle. Maybe they don’t remember it, but they know that there’s this radius thing involved. And so by not allowing them to sort of quit until they actually are in a perfect circle, the only way to do that is you have someone stand in the center, and you take another piece of clothesline to measure your radius, and you move everyone in and out as appropriate. So, that activity of physically making the circle and by having to have that person in the center, and that radius gets them to say the definition of the circle properly, first of all, but they get to experience it in a way that they don’t get to otherwise. And that’s an activity that I never would have thought of without going to Tensia Soto’s session at my first Project NExT meetings.

John: It is certainly safer than giving them all compasses with sharp points where they can stab each other, which was how people used to do it.

Sarah: We still do compass and straightedge constructions in geometry, but again, that doesn’t actually help you really understand what the definition is. I think doing this physically actually helps them understand why a compass works. I know that sounds silly, but it really helps make those kinds of connections. I have another activity where we take clothesline and I make a triangle on the ground, and I make them walk the interior angles of the triangle and you spin 180 degrees and it, again, helps them experience that the sum of the internal angles of a triangle is 180 degrees. And again, that’s something that, the first time I did it, it was baffling because first of all, it’s hard to turn the interior angles. Your instinct is to turn the exterior ones, but you end up backwards. From a geometry standpoint, it makes sense, but somehow that physical aspect just really changes things.

John: It makes for a much more memorable experience, where they’re seeing things from a different perspective. And I think that’s really useful.

Sarah: I agree. That’s why I do it.

Rebecca: Does anyone else want to chime in about how having so many fellows from Project NExT has influenced the larger department? Because you’re not just five people in your department, you’re how many?

Sarah: There’s 14 of us tenure/tenure-track now. I do think it’s changing the way some things are done. It’s slow going. I think everyone would concur with that. Jessalyns’s smirk is definitely confirming that. It’s slow going, some of us would like change to happen faster. But, I do think change is happening. I think there’s a lot of respect from our colleagues that we are trying new things. I think a lot of them have a “You can do what you want, but don’t make me change yet.” But, I think we’re starting to get them a little bit, too.

John: If your students are more successful, that often convinces people and sometimes when students say, “I did this in this other class and was really helpful,” that’s often really persuasive to other faculty. But, it’s convenient that you had so many people all come in at once, because that’s not typical in most departments that have such a large cohort, in a short period of time.

Sarah: We have had a lot of retirements, one right back on top of each other. So, we have had an influx of young faculty in our department, which…that alone…to have so many in this program as well. Definitely.

Rebecca: I think it really helps to have models of ways that you can do things because if you didn’t learn using these methods or you didn’t have exposure to that as a student, you have no way of knowing how those really play out unless you have examples. So, it sounds like Project NExT played that role for you, but then you are playing that role for other faculty in your department.

Jessalyn: Thinking about department culture more broadly, not just among discussions and relationship among faculty, but in terms of the student experience, and this engagement that we’re having from our majors and the sort of activities that we’re involving them in. I think there has been a Project NExT influence there as well. Sarah, you and John started the Putnam Competition before I came even and a lot of other conversations and gatherings have come out of that, like we’re getting together with our majors and talking about preparing them for graduate school if that’s something they want to do. The math club or other organizations have taken on a different role in the department and I think a lot of that comes out of some of the ideas in Project NExT, like hearing about how another department celebrates their students participating in something like the Putnam Competition. But, it also comes out of the relationships you build in an active learning classroom and the way that we connect with students when we are trying new things. And we’re being honest with them and saying, “Hey, I’m trying something new. And I’m going to want your feedback.” The community that you build in a classroom flows into the community that we support and foster as a department.

Zoe: So, it’s a bit hard for me to talk about departmental culture change in the one month that I’ve been there not having seen it before I did Project NExT. But, I can certainly talk about how the department seems different from other departments, just in the willingness to embrace new ideas. And there’s also a sense that these ideas are just supported. Even if we haven’t had an explicit conversation, I know that there will be support for trying something new that was suggested in Project NExT. And it seems, when it comes time to make policies, that we have almost a majority just of Project NExT people. Obviously, we need a couple more people, but there are other people who haven’t participated in the program who would still support these sorts of initiatives. Knowing that that base of similar views is there, makes a big difference in what sorts of ideas we would even suggest or consider.

Sarah: I think a lot of our Project NExT fellows have also been very active with doing undergraduate research with students.

Rasika: I think even like talking to colleagues. For me, like I have a personal experience, because my husband is also a mathematician and teach at SUNY Oswego. If I learn something new, I share with him of course, he’s not a Project NExT fellow, but…

Rebecca: So, it sounds like the program’s working really well. You’re all really excited about it. It sounds like it’s engaging all of you. So, glad that you’re able to share it with us.

Sarah: The MAA has definitely done a lot to support improving teaching in mathematics and I do think it is a program that other disciplines could look at and possibly model. I will say they have put a lot of money and a lot of investment into making this a success. It is well run and has been well funded, which is a testament to how important professional organization views it.

John: We always wrap up by asking, what’s next?

Sarah: Well one thing that’s next is we’re trying to get one of our other new faculty, his application was rejected last year. We’re also hiring two people, hopefully this year…So, possibly trying to send them next year as well.

Jessalyn: Another immediate thing that’s next is that our two current NExT fellows will be attending the joint math meetings in January and maybe organizing some Project NExT sessions or at least attending some sessions.

Zoe: I’ll be helping to organize a session on getting started in math education research, which I was made part of because I said it was something I wanted to do, but it’s not something I have any background in. So, I’m finding it a bit of a challenge to assist in this organizational process. But, I also, possibly for Math Fest next summer, helping organize a session on reducing math anxiety, which is something that a previous NExT fellow who I follow on Twitter help organize this session. So, having attended NExT, I think, gave me the confidence to respond on Twitter to this senior mathematician and say, “Oh, yes, I’m interested in this topic.” And so that will come later. And that’s something I actually feel like you could contribute to in a meaningful way, unlike math education.

Rebecca: Well, thanks so much for joining us. This has been really interesting.

Sarah: It’s our pleasure.

Jessalyn: Yeah, thanks for having us.

Zoe: Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

John: Editing assistance provided by Brittany Jones and Kiara Montero.

95. Specifications Grading

Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, Dr. Linda Nilson joins us to explore how specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

Linda is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of Specifications Grading: Restoring rigor, motivating students, and saving faculty time as well as many other superb books, book chapters, and articles on teaching and learning.

Show Notes

  • Linda Nilson – Director Emeritus of the Office of Teaching Effectiveness and Innovation (OTEI) at Clemson University
  • Nilson, L. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Stylus Publishing, LLC.
  • Nilson, L. “80. Self-Regulated Learning.” Tea for Teaching podcast. May 8, 2019.

Transcript

John: Faculty often find that grading student work is a stressful and time-consuming activity. Students sometimes see grades as a subject of negotiation rather than as an assessment of their learning. In this episode, we explore how the use of specifications grading can save faculty time while motivating students to achieve the course learning outcomes.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer.

Rebecca: Together we run the Center for Excellence in Learning and Teaching at the State University of New York at Oswego.

[MUSIC]

John: Today our guest is Dr. Linda Nilson. She is the founding director of the Office of Teaching Effectiveness and Innovation at Clemson University. She is the author of many superb books, book chapters, and articles on teaching and learning.

Rebecca: Welcome, Linda.

John: Welcome.

Linda: Thank you very much. It’s great to be here.

Rebecca: I think it’s more of a welcome back, right?

John: Yes. Welcome back.

Linda: Welcome back. Yes, yes, yes. Good to be here again.

Rebecca: Are you drinking any tea today?

Linda: As a matter of fact, I am. Yes, yes, yes. Of course I had coffee this morning as well. But I am drinking tea. I am drinking a berry tea, but it also has black tea in it. So it’s still a bit of a stimulant anyway, but it tastes really good.

Rebecca: Sounds good.

John: I have ginger peach green tea.

Rebecca: And I have English breakfast.

John: In an earlier podcast we talked with you about your work with self-regulated learning and one of the topics that came up with that was specifications grading.

Linda: Yes.

John: So we’d like you to tell us a little bit about your book, Specifications Grading: Restoring rigor, motivating students, and saving faculty time… especially that last part. [LAUGHTER]

Linda: Yes, saving faculty time. Yeah, I figured that would help sell the book. But it’s true. It does save faculty time. And one of the things that inspired this book was just my hearing so many complaints from so many faculty over the years, about grading and the aftermath of grading and returning grading material to students. And the constant steady stream of students trying to get another half a point, just arguing… just conflict… constant conflict… students being stressed… faculty being stressed… faculty getting larger and larger classes with less and less help… fewer and fewer TAs, if there ever any TAs. And I got tired of it. But there was a part of this that I did not invent. I heard it from a faculty member in the School of Management. And she was doing that pass-fail grading. It was saving her tons of time. She had huge classes, online classes. And she just invented this and she was also sick and tired… getting complaints from students and students not paying attention to her feedback, which of course, took her hours and hours and hours to write and return. So I took some ideas from her. But I also wanted to tie grades somehow to outcomes. And this is where another aspect of specs grading comes in. And that is with respect to bundling assignments, or turning them into modules or whatever. But I prefer the term bundles because it’s much more universal. So anyway, this was a solution to a problem. And that’s what a lot of my work has had to do with, making the faculty members job easier and more rewarding.

Rebecca: I think that you’ve been spying on me for the last many years. [LAUGHTER]

Linda: Yes I have. Yes I have. [LAUGHTER] I’ll admit it.

Rebecca: I knew it. I knew it. So I think John and I are both really interested in the idea of saving time, as are many faculty. But you also talked about, in your book, the history of grading… how its evolved… the 4.0 system. How is this different? And how does this relate to the history of grading?

Linda: Well, let’s look at the history of grading first. Grade started, well everywhere, in 1783. It was Yale’s idea. And what they started doing in that year was an achievement-based student classification system. They were not using As and Bs, what they were using were Latin designations of like optimi outstanding and pejores for failing, like as in pejorative, right? Anyway, then in 1800, Yale dropped the Latin designations and started using numbers 0 to 4. Sound familiar? But that was Yale doing that now. In 1850, the University of Michigan initiated grades, but for them, it was strictly pass/fail, and it only took 50% to pass. So we talk about grade inflation now… look backwards. [LAUGHTER] Mount Holyoke, though, just a few years later set passing at 75%. Harvard, also a little bit later, invented the A to F system, but passing was only 26%. So if you were wealthy enough, you had to know less, right? Okay. But anyway, that’s where ultimately grades came from before 1783. This was in Europe. And this started hundreds of years before our notion of grades. There was something… it was like a Jeopardy game, where graduates… or graduates to be… the hopeful graduates… were answering questions in a tournament style, but the stakes were really high. And so yes, if you are winning throughout, you were really showered with honors. But if you were at the bottom, you lived years in shame. It was terrible. We talk about high stakes. Oh my. So anyway, grades were invented after universities. Socrates didn’t talk about grades, right? It’s a relatively new invention for sorting students.

Rebecca: How does the specification grading relate to this letter system or this 4.0 system?

Linda: Well, it kind of takes a break away. Because first of all, all this grades that I was talking about was with respect to courses, there have been pass/fail courses. Sometimes they worked well, sometimes they didn’t work so well. They work pretty well in medical school ‘cause we’re dealing with a highly motivated students who really understand the need to know. But other than that, most students would do the absolute minimum to get their C-… whatever, it didn’t work very well in terms of like motivating students to learn, they learned the absolute minimum. With specs grading, the pass/fail is within the course, the assignments and tests in the course. But, you don’t pass with a C or C-. You set the passing level at what you would regard as a B level. And this is what restores the rigor. I think we have been sold a poor bill of goods, when we don’t set our students to high standards. We say, “Well, maybe they can’t do it, and then they’ll get…” They can do it. Come on. They’re just not doing it because of our partial credit system and our point system ‘cause they can always get another half a point, right? …just by wearing you down. [LAUGHTER]

Linda: Really, they do and they know it works, it worked in high school, so it works now. Now it may have worked the previous semester, these students aren’t stupid. And they can do what we ask them to do for “B” work… certainly at the undergraduate level, or actually at the graduate level as well. They could do it. But they choose not to, because they can survive otherwise. They could do well enough to pass or get their “B” or even get their “A.” So why should they sweat it, they can always get the partial credit. And students get partial credit. It’s almost like going back to the University of Michigan 50% passing for assignments and tests. I mean, we pretty much lowered the bar, because we want so many people to get over it. Well, people are getting over it without preparing… without doing a decent job. And they’ll spend 10 minutes… 20 minutes… the night before on an assignment knowing that no matter what they do, they will get partial credit and they will pass. So again, they’re not stupid, it’s just that, for them, college is a game that doesn’t have much to do with learning. That’s not where the focus is, and at least with specs grading, there is a lot more focus on learning, because you are tying the assignments and tests to student learning outcomes. And that’s a really nice part of the system in that those grades, the ABC, mean something in terms of outcomes achievement, and these, in turn, might be tied to program outcomes as well. So all of a sudden, you’ve gotten rid of an entire step at the departmental level of having to measure program outcomes. And why do you have to do this? Because accrediting agencies know that our grades don’t mean much of anything, as far as learning is concerned, as far as what they’re interested in, which is outcomes achievement. And outcomes achievement, it’s an up-or-out sort of thing. You can either do something or you can’t, and as long as there are standards set for that, and certainly the accrediting agencies want standards set for that. So let’s say “Okay, so your goal is a student can write a good quality, maybe not great quality, but a good quality business proposal.” Okay, fine. So if that is your outcome, what really does that entail? What exactly are you looking for in terms of a very good business plan? And that should be incorporated in your directions to students. But we don’t articulate that, do we? We really don’t. We speak very vaguely, “Well a business plan should have this and here’s your rubric.” Well, first of all, students often don’t understand the language, but is simply not detailed enough such that all students can understand the directions and actually follow them. We need to put more detail. Usually what we assigned undergraduates is some sort of a template or a formula to do this. And we’re not talking about that. We’re not sharing this template or formula with our students. And that’s what we need to do… not to say that you can’t allow for creative work. Matter of fact, with this system, you can and you don’t have to worry about tearing your hair out in different ways. “Well, I don’t know how to grade a movie.” Well, no, but you can talk about certain qualities of that movie, as in just simple things like the length, or perhaps the number of scenes that you want to see. And certainly they’ll be a learning goal and a communication goal connected to it. And that’s pretty much all you have to do. For most of our assignments, they are formulaic, but students don’t know the formula. And we need to tell them, and those are what you would call our specs for an assignment. So that’s really what we need to do. So all our work is up front and laying out what those specs are. It’s like a one-level rubric. But we don’t have to worry about different levels of the rubric, where there are four-level rubrics and five-level rubrics. We don’t have to worry about that. All we need is one. So we can afford the time to actually specify what we’re looking for… what that template involves.

John: And then the focus is helping students reach the standard, rather than negotiating with them over the grades for what they’ve received.

Linda: Exactly, exactly. Yeah, forget negotiation. There’s some things that ought to be non-negotiable. I mean, we’re not sloppy when we grade, I know, we get sick and tired of it. But we are so diligent in our grading. And then we have a line of students outside our office saying, “I deserve another two points on this because Susie said the same thing. She got 12 points, I only got 10.” …like write an essay about it… justifying it.

John: One thing I really like about it, though, is that right now we have these two levels of assessing students’ learning, we have these complex assessment plans for each department. And then we have a grading system, which often bears little resemblance to the assessment. And this is making the assessment transparent and obvious to students. And it’s forcing both faculty and students to focus on the learning objectives for the course.

Linda: Yes, as we should be, you think about it, what does it mean, when a student gets an A? Does that mean that that student has achieved all the objectives? all the outcomes at the level we want to see? Well, maybe for some of the students who get an A. What does a “B” mean? Now we’re starting to get into really ambiguous territory. Does that mean that the students sort of achieved or barely achieved all the outcomes or maybe achieved some really well and then didn’t achieve others? And a “C,” forget it. You can’t tell what’s going on there, especially given the way we give out “Cs” these days. So maybe the “C” student achieved one outcome well, but which one? Yeah, I mean, no wonder accrediting agencies pay no attention to our grades. They don’t even want to know them. And I can understand why, if they’re focused on learning outcomes. And so yeah, we’ve got this whole extra level of work that we have to do and the department chairs have to do and the Provosts have to do. I mean, who needs it?

Rebecca: Can you talk a little bit about the relationship between specs grading and contract grading, because it seems like they have some things in common?

Linda: Yes, they do have something in common. With contract grading, students individually work it out with a faculty member as to exactly what they’re going to do. And if they do all those things that they will get there “A” because I’ve never heard of a student contracting or a ”B,” it just doesn’t happen. Now, contract rating goes back to the 60s, and the first half of the 70s. And what was happening was since there was this individual relationship, first of all, faculty didn’t specify enough about what they wanted… what constituted “A” work… same problem we have had up to now… and faculty weren’t all that much different then, in fact, they probably gave less guidance. But in any case, there were the specifications. And then faculty and students would develop this relationship, this individual relationship. And it became particularly difficult for the faculty member to give anything but an “A” to the student. And the student would have thought, “Well, why didn’t you tell me to do this to get my ‘A,’ because that’s what I contracted for.” So contract rating was: 1. sloppy, but 2. highly individualized. With our class sizes, we can’t even talk about contract grading. And because it was something that was mutually established, between the faculty and the students, and specs grading is nothing like that. This is all in the faculty members’ hands. Well, at least, setting out what is required to get the “As” and “Bs” and “Cs” and “Ds” . But students choose what grade they’re going to go for. And according to what grade they believe that they need, but also according to what kind of a workload they want to shoulder for this particular course. And you know, maybe all they need is a “C.” And you know what, that’s okay. And we don’t look down on that student if the student says: “Well, all I need is a ‘C.’ We do otherwise. We say “Wow, this is a lazy student.” That’s what we’re thinking in the back of our heads. We have negative thoughts about students who settle for low grades. But with specs grading, you don’t have to feel that way. This student chose a seat for whatever reason, I don’t care. My course is not in that student’s major. So I’m not going to take it personally. So, that’s nice. But the fact that that’s making that choice, gets rid of all kinds of grading complaints, or that thing at the end of the semester: “What can I do to bring up my grade?” Well, it’s very clear what you could have been doing. If you went for a “C” now you could have done these additional assignments, or taking this additional test to get a “B.” But you didn’t do that. Now, you’ve got a week left, maybe you want to try to do that. But don’t ask me, I laid out the contract, I laid out the terms, I laid out the specs at the beginning of the semester, and you make your choices. This whole thing makes students feel a lot more responsible for their grades. They made the choice. And we respect that.

Rebecca: So I think a question that would come up for many faculty is you’ve laid out that certain assignments need to be accomplished to get even a “C…”

Linda: Yes.

Rebecca: …and what the specifications are. What happens when a student struggles to meet those specs?

Linda: Oh gee. Well, now when you say struggle, you can struggle before the assignment, or you can be disappointed and angry after the assignment. So if you’re talking about struggling before and say, “Professor, I don’t understand these two specs. I don’t even know if I can do it. Could you give me some more guidance?” No problem, and then because the stakes are higher, you get credit or no credit whatsoever… doesn’t count towards your bundle… doesn’t count towards whatever grade you’re doing. So you’d better figure this out. Now, afterwards, don’t meet the specs. Okay. And obviously, you didn’t come to see me in advance, did you or otherwise, we would have hashed this out. Anyway… so you didn’t pass? So you missed these two specs. All right, well gee, that’s most unfortunate. But there’s this system called tokens… or I call them tokens, but you can call them… I don’t know, you can call them pigs in a blanket for all I care, it doesn’t matter. They’re like get out of jail free cards. I’ve seen them called hail Mary cards. In a geography course, they were called globes, but anyway, they are opportunities to either redo an assignment that didn’t meet the specs, or to get a 24-hour extension. And there might be some other things that you make up along the way if absence counts against you in your course, you can get out of absence. But let’s say it’s for redoing this assignment, which is really important early in the semester, because students are not going to believe that you’re going to grade them pass/fail, because this is all new for them. And they go, “Well, how can this be…” and “There must be some sort of partial credit or something…” They won’t believe you. So there will be a number of students who will need this token or get out of jail free card to redo the assignment. It’s up to you, as a faculty member, as to how many get out of jail free cards or tokens that they might have. Three is a nice number. I like it, but some people give five. Some people give two. But I think it behooves you to have some reasonable number of tokens with specs grading, because that takes the pressure off of students; it allows them to screw up at least a few times. So this is something that students can get second chances out of this because the stakes are higher than typical. But the thing is, when you’re grading these assignments, all you’ve got to do is check the specs that weren’t met, and say, “This is why your assignment failed.” Now, if you want to give additional feedback, by all means, I’m not going to stop you. But I want you to have a life. That’s the only thing. And you can give positive feedback as well. And then you know what, you’ve passed something, then you get positive feedback from your instructor that you figure, “Wow, the instructor gave me this positive feedback out of the kindness of her heart, because she really cares about me and my success… told me how I could maybe do better in the future.” First of all, I’m going to read this, because this is meaningful feedback. This is not justification for having taken off points, because that’s what most of our feedback is about: why you didn’t get full credit. And now this is actual substantive learning feedback. So, it decouples feedback from evaluation in a way, but students get the chance to do it over and now that you have their attention, that let’s say they didn’t pass a certain assignment, that they can come to you and say, “Okay, how should I do this better?” Oh, okay. Isn’t this lovely? So the worst that will happen is you’ll have a conversation with a student about how to improve his or her work, that’s the worst that will happen. Isn’t that wonderful? [LAUGHTER] And sometimes students know, they just got lazy.

Rebecca: How would you suggest structuring specification grading for a class that’s more project based or even collaborative work?

Linda: Oh, sure. Whether it’s projects, or papers or whatever, again, it’s a matter of specs. Oftentimes, with our projects, that we might allow our projects different media, what we have to do is lay out specs for each media. Or say, if you can think of a different media, you can do a film, you can do a play, you can write pamphlets, you can write a paper if you want to, or you can do any of, let’s say, a half a dozen things. And let’s say if you want to do something else that’s not on the list, come talk to me, and we’ll work it out, we’ll work out some specs for it, too. So what you have to do is you have to lay out specs, but oftentimes the specs that you have to lay out, and students love creative assignments, is you have to lay out, let’s say, length, and so you might say, “Okay, if you’re going to go with a film or video, I want 20-mintues of that. Going to go with pamphlets? I want at least four pamphlets, I want each pamphlet to have at least 250-words in it. If you’re going to go with writing a play and performing a play for your peers, I mean, wow. But I do want it to be a half an hour long. And I want it to involve everybody in the group, everybody’s acting. And if you’re going to write a paper, I want it to be at least, let’s say, 2000-words long… or maybe that’s too long, whatever… and you lay it out, and I want it to follow this kind of an outline.” You can lay out specs for different kinds of media, and let students run with the project. Now a design project, what you might allow students to choose is exactly, let’s say, in architecture, what kind of a building they’re going to design. Let’s say, “Okay, you can design a residence, single family, a separate dwelling, but I want it to be at least four bedrooms, I want to be two stories, this and that. And you could do an office building too. But I want this to be at least two stories, and I want it to be so many thousands of square feet.” And then you let them run with it. And you know what, we know from our history, and from a lot of publications, that students run with this sort of thing. So I’ll give a little guidance and off they go, it’s a beautiful thing. Now in terms of projects having to do with group work, I don’t know that that specs grading will make group work smoother. [LAUGHTER] And I’ve talking about out-of-class group work. If you want to play marriage counselor, you just go right on ahead. But you don’t have to. I mean, there are some people who say, “Hey, you guys work it out on your own, I don’t want to hear about it and you can fire freeloaders. And if you are doing all the work, you can resign from your group and any other group will be happy to pick you up. But all this has to be done by the end of the fifth week,” whatever. So I don’t know that we’re going to get rid of those problems. [LAUGHTER]

REBBECCA: I think my question was more about the high-stakes nature or the long-term nature of projects.

Linda: Yes.

Rebecca: So would you recommend doing scaffolded specs or something like that to help make sure they’re on track?

Linda: Well, yes, what you need to do, as with any major project, you need to divide it up into smaller tasks that students can be at accountable for along the way, and maybe even get feedback for. Now, you can give feedback… your peers… people in other groups… can give feedback. Because after all, we’re doing criteria referenced grading, we’re not grading on the curve. So there are different ways for students to get feedback. But in any case, this divides a massive task into little pieces. So maybe starting out with, “Okay, by the end of the third week, we want your group to have a literature review put together.” And we put specs for the literature review. “And we want so many references. And we want so many of them in the last five-years,” And what have you. So we can even put the specs, I even specify specs for the format that we want: APA format, or ASA format, or Chicago, whatever it is that we want. That’s something that we can put in the specs if we want to, and that way students will actually proofread. Now I think we ought to be somewhat tolerant with respect to format anyway, we ought to allow three little errors in format. Because you know what, when we submit a paper to a journal, they’re tolerant of us, right? [LAUGHTER] I mean, you can’t be sloppy, but if we put a period instead of a comma, hey, we are forgiven. And we’ll catch it later on the proofs. So we need to allow some errors on something like that. But anyway, yeah, we divide up a big task into pieces. And then students get feedback on the pieces. And that’s also a good way to find out early about who are your freeloaders, who’s not doing the work early, so they can get rid of the freeloaders if they need to. Here’s another thing that you can do with specs grading, when you do bundles, you know, the project, the big project, you can make that big project required for an A, period. And then you will only have groups with students who are going for As. So it depends on the size of your class, of course. You might wind up with only, let’s say, three groups who are going for As. Or you know, the number of students that will fall into three groups. Well, guess what, you’re going to get some really fine projects, you’re not going to have to worry about it, and they might not be freeloaders. But anyway, in the examples in my book that a lot of faculty members and I recommended as well, that group projects be reserved for getting an A, for the A bundle. And other assignments be in the B bundle and C and the D bundle for that matter, which might be very minimal. But hey, if you want to a D, far be it for me to stop you. So, anyway, that’s another way to solve the problem, you just put those projects in just for the students are going for an A.

John: You’ve mentioned bundling a few times in terms of grades, could you talk a little bit about that and how you can go from the specs grading to the course grades that are assigned in the course.

Linda: Okay, there are two ways and the first way I’m going to just mention and not recommend. But you can keep your point system, and you might have some haggling over grades. But in any case, you can keep the point system and say, “Okay, my course is 100 points and if you get 91 points or more, you’ll get an A.” And okay, fine, but the bundling system allows students to choose their grade. So what you do is, you set up clusters of assignments and tests that students have to pass at that B level to get credit for their bundle. So you set those up. And the easiest way to do this, so you got a D bundle, is pretty minimal. And you can tell students, “Look for your D, here are the only outcomes that you will be able to achieve if you go for the D.” Some students say, “Hey, I don’t care, not in my major.” Fine. So it’s minimal work, maybe it involved just passing tests at let’s say, a 70% level, and maybe doing a couple of little written assignments along the way. But, they all have to meet whatever the specs might be. Whatever specs you set out for those assignments… And C bundle, “Okay, you got to do everything that you do in that D bundle, plus there are extra assignments as well.” And so maybe what you are having students do, and I’m just making this up as an example, is the students have to turn in 15 out of the 18 reading assignments that you have, they have to turn in a typed up outline on a particular reading. This way, at least your C students are going to do the readings, right? Because at least most of the readings, almost all the readings, and you’d give your teeth for almost all your students to do almost all the readings, right? But that’s all they’ve got to do. Maybe for a B you have to do all the requirements for a C, maybe even more of those reading notes, or something different. Perhaps keep learning journal, keep a journal on how you are learning, what you’re having trouble learning, the different strategies you are trying to learn the material, and it’s going to be collected. But you really have to specify what questions you want students to answer. So that will be collected maybe four times during the semester, and you might collect some every week or so and look at a few of them. But again, you’re just looking for the answers to the questions. That’s what you have to do for a B. You’re going to get a good handle on those readings and you’re going to find out how to best learn this material. For the A, you got to do everything for the B, but you have to do some sort of a group project where you’re going to learn, let’s say application, analysis, evaluation, what have you, whatever higher thinking levels that you have in there. At the C level, that’s pretty low. But good lord, at least they’re understanding the readings, at least they’re reading the readings and presumably understanding it. And you know, you can spot check some of them to see that they are understanding and if they’re not, you can help them with that. And as soon as they’re learning how to learn for that B, but students are learning how to analyze the material, even higher-level learning outcomes for that A. Now if your student learning outcomes are dictated by a professional accrediting agency, then you’ve got to put all of those outcomes in what’s required for a C, or maybe your institution considers D passing, some institutions don’t. Whatever you consider passing, that’s where all of those outcomes have to go. B students and the A students, they will achieve even more outcomes. If you’ve got an outside accrediting agency telling you what your students have to be able to do, then guess what? All the students that pass the course have to be able to do these things. And you’ve got to set out the specs accordingly. There are some accrediting agencies that layout good outcomes with active verbs. And other accrediting agencies, they give you crummy outcomes where you’ve got to essentially rewrite them so you can assess them. But I’m not going to get into that. [LAUGHTER] Did I answer your question, I hope?

Rebecca: Yeah, I think a lot of your examples focus on a more traditional class and not necessarily a lab or a studio kind of class where there’s a lot of class work that happens inside of class. What might a bundle look like in a situation like that? Where a lot of the learning activities are happening in person rather than for homework. So, for example, I teach studio classes that meet for six hours a week, and they presumably do about three hours outside of class. Versus the reverse, which many people have and labs are similar.

Linda: What area are you in?

Rebecca: I teach graphic design.

Linda: Okay.

Rebecca: But it would be similar to something that would happen in the sciences too and having to do lab.

Linda: Yeah, to an extent, it depends on whether it’s a decent lab or not. [LAUGHTER] Some labs are a waste of everybody’s time. But anyway, what you have to do there is, you’ve got projects, right? …throughout the semester, different things that students are doing. but how many projects do you have in a semester?

Rebecca: Usually, it’s like four big projects, or three.

Linda: Okay, well, now this sounds very, very radical. But what if for a D, you only had to do one project? …and I mean, at that B-level that you set to pass. What if for a C, as a student, I only have to do two. Now, for the first two, because I presume this is somewhat cumulative. For a B, I only have to do three projects. But an A, of course, I have to do all four, if there are four projects. Now, what does this mean? Oh, by the way, do you have tests?

Rebecca: No.

Linda: Just curious. Okay, it means that assuming your D student does passable work, that first project, you can say bye bye after that first project is done. If the person wants to sit in… you can’t, you know… it’s a free country, right? Well, not really. But they can sit in if you want them to, that’s fine. For a C student again, you can say bye bye after the second project, assuming they do it at that passable level. And by the end of the semester, you have only you the most motivated and committed students that you have to be concerned with. And you can give them some very challenging work. Now, I don’t know how you feel about that. But that’s the way you could do that. Now, you could also bundle it differently. You can expand the number of projects that you have as well and simply make them shorter if that’s what you would be required. But I don’t know how you react to that.

Rebecca:I think my accrediting agency would react to that in that they have to put a certain number of hours in. [LAUGHTER]

Linda: Okay, okay, so your accrediting agency is time focused.

Rebecca: It’s part of it. Because presumably with more practice, you get better.

Linda: Well, presumably. [LAIUGHTER] …practice with feedback, you get better. But yeah, if we are committed to this hours business, that makes things very unruly, everybody’s got to put in the same amount of time…

Rebecca: In class anyway.

Linda: Oh, in class, okay. That’s very unruly, with respects to specs grading because specs grading isn’t about time. I mean, you can recommend that such and such assignment should take you at least three hours for it to be passable. But other than that, that’s your recommendation for a certain given assignment. But yeah, that makes it very unruly when your accrediting agency says, “Okay, they’ve got to spend so many hours in class and not just looking at their phone.”

Rebecca: Yeah, I mean, we do critiques and things like that. We provide a lot of feedback. But yeah.

Linda: Yeah. Okay. You do critiques?

Rebecca: Yeah lots of critiques.

Linda: Yeah, like art classes and things like that?

Rebecca: Yeah.

Linda: Okay. Well, that’s nice. But now, you could hold your B and C students to doing critiques on all the assignments, just so they can put in their hours. But they don’t necessarily have to do the assignments, at least they can see models. But again, I don’t know if your accrediting agency would be happy with that either.

John: This may not exactly be specs grading, but could you give, say, different levels of activity in each of the assignments? If you have four-projects, you could have one bundle in that project that would give a C, another that would give a B, and another that would give an A, changing the scope of the projects and it would be specs grading within that scope.

Linda: Yes, that has been done and I’m not sure about what kind of projects you’re doing. But yes, you could do that. And you could set different specs for each level. So D students, lets say, have to do a D-level project, which is not nearly as time consuming and not nearly as high a level cognitively, it doesn’t demand that much, doesn’t demand much time, it doesn’t demand that much thinking. And so you could do that and that way students would put in their time, and as long as you can set out the specs for those levels. Yes, absolutely you could do it that way.

Rebecca: Yeah.

Linda: But yeah, accrediting agencies can really get in the way of flexibility. Rebecca, would your accrediting agency go along with that?

Rebecca: With scope, probably. Yeah.

Linda: Yeah. Okay, good. I want to tell you about a course that does this, it’s in computer science, they have students writing programs, and there are six projects that students have to do, they do these individually. If you are going for a C, you get some pretty easy projects to do, pretty easy problems to solve. If you’re going for a B, well you get sort of intermediate-level problem, you don’t do the C one. Because if can do the B ones, you can do the C ones. Now for an A you get some much more sophisticated ones. But still, for every level, you only get six. So there is that, that as long as you can designate level of difficulty, or for that matter, breadth of knowledge, if you can designate that… of course, usually people can do that… that you can make levels out that you can make different bundles out of that. This is another little interesting take, in this particular course, if you are late handing in your projects, that you’ve got to do two more projects at that level. So in other words, you are penalized for lateness by having to do more work. Now that really hits students where it hurts. And if you’re like super late, you got to do yet two more. So any way that keeps students on their toes. [LAUGHTER] Anyway, so in this particular course students are rarely late. So yes, there are ways that you can do that just in terms of level of complexity or level of cognitive operations demanded.

John: How do you explain this to students, if you’re going to introduce a specs grading system?

Linda: First of all, you’ve got to sell specs grading, because they’ll look at you like, “Huh?,” which is fine. And you should explain this, but you can tell them, “You know, I’m going to hold you to higher expectations.” And we know that holding students to higher expectations feeds into student success, student learning, but also student success in general. You could explain to them the concept of andragogy. In other words, that pedagogy really has to do with little kids. Andragogy has to do with adults. Saying, “I’m going to treat you like an adult.” And students like that kind of thing, you might be choosing the kind of project you’re doing. It’s easy to build choices into this. “This will be a safe but a challenging environment because you know exactly what you’re supposed to do. I’m going to give you the descriptions of what you have to do for the assignment to pass in advance and in detail. So you almost can’t screw this up unless you don’t read the directions or pay attention to the directions.” So you want to emphasize the choice and control over their grade that they’re going to have. You want to tell them about tokens, we’ve got some wiggle room, if you will. Now these tokens, by the way, are not physical, you make them physical students will develop a black market, [LAUGHTER] believe me, but virtual and you keep track, or you can ask for an extension, but you don’t ask me for an extension. You’re late, I just take a token away. That’s all. We don’t have to talk about it. I don’t care why you’re late, this is just the way it’s going to be. Just like your boss won’t care why you are late or why you are absent, it counts as like a holiday. Now we’re going to tie what your grades is to what you are learning. If they look at the syllabus at all, everybody’s going to know what you have learned in this course. So that’s how you sell it to students. Now students will need to be reminded about how the system works, because this will strike them. “This is so weird.” Even if they initially like the idea, they still need to be reminded a few times because it is really strange. According to my research, students way prefer this system to traditional grading. And one of the reasons why they do is because we give better direction, we tell them what we want, we give them the formula, we give them the template. or we give them tons and tons of freedom to meet the specs. Especially with students when we talk about length or whatever, length means depth to the students. It doesn’t mean that necessarily to us. Students really like this because they feel way more secure in it. And they do like the element of choices that are built in, that’s motivating for them. And they do learn more. And if they are more motivated, they’re likely to do more of the work at a higher level, they are more likely to excel. And the A students are going to do A work anyway, because they don’t know any better. All they know is A work. And there’s a sense in which they want us to love them. I mean, really, so they’ve got this strong sense of loyalty towards us and they really want us to respect them. And they’re used to feeding off of that they’ve been doing it all their lives. Don’t worry about them getting sloppy, they’ll never get sloppy on you. [LAUGHTER]

John: If someone wanted to transition to specs grading, how should they get started?

Linda: What you might want to do if you’re switching, transitioning to specs grading, you want to look at your assignments, and for that matter, look at your tests, like what you have in your tests. You want to look to see what can be transferred into pass/fail. You also want to look at your tests, if your tests are very objective, and you’re relying on a test bank, remember, it’s so much easier for you to grade with the specs that you can start assigning more written work or design work on tests, what have you. But definitely higher-order thinking types of questions and you will have the time to grade them, because you’re going to lay out the specs and without giving away the farm, you will tell your students the specs of your essay questions in advance, and they will study accordingly. But anyway, you want to look at your outcomes and then you want to identify the cognitive level of your assignments and your test questions which again, you might want to change. You want to be able to group your assignments and tests by that cognitive level so you can develop bundles. It’s kind of a radical way to do it, but it makes your life so much easier once you have those bundles. Because all you’ve got to do is say, “Okay, let’s see. So I had four assignments in this bundle, this student passed them all, hey B. It’s so much easier, you just have to be sure that you set out the deadlines for the different assignments in the bundles. You don’t want everything being submitted to you in the last week or two. So you still have to have your deadlines along the way. Just warning you about that. I want to refer people to my book for examples of courses, and not just my courses either, that have been specs graded. And they cover a range of over a dozen different disciplines. So they can see all kinds of examples.

John: That’s a great resource for those who are considering moving to specs grading.

Linda: Yes.

Rebecca: And it also sounded to me like a couple of cautionary tales that you have about transitioning are about time and making sure that you spread things out…

Linda: Yes.

Rebecca: …but also being very clear in what the specs are, and then following through with whatever you said.

Linda: Yep, yep. And if you didn’t lay out enough detail, well, there’s always next semester. [LAUGHTER] But you can’t change those specs in midstream. “Oh, I wish I thought of this and then start grading students.” No, no, no, no, no. But again, we live and learn, right? We’ll get better the second time.

John: We always end with the question. What are you doing next?

Linda: Oh, what am I doing next? Well, what I’ve been doing is I’ve been doing a lot of traveling, doing keynotes and workshops… oftentimes on self-regulated learning, sometimes on specs grading, and sometimes on any number of different topics having to do with either teaching or academic writing, or something like that. So I’m going to be taking a little bit of a break from that until the end of, or late, July anyway and then things start up again. But that’s kind of nice. I don’t mind that because this keeps me semi-retired and that’s the way I want to be. A semi-retirement is Nirvana, just letting y’all know. [LAUGHTER] So, that’s what’s next for me, that and to finish my tea.

Rebecca: Sounds like the key to semi-retirement is the semi not retired part. [LAUGHTER]

Linda: Yes, it is, it is. Your brain is still working and you still got your thumb in the pie so to speak. And it feels good because you’re not under that same 60 hour a week pressure that you otherwise have with a regular job and then doing these other extra things on the side and it doesn’t work real well.

Rebecca: Thank you so much for joining us.

Linda: Well, thank you for this opportunity. I normally say… autograph books and things like that… happy teaching, but I want to wish everybody happy grading. [LAUGHTER] An odd phrase right? Happy grading. [LAUGHTER] What an odd phrase, right? Happy grading!

John: Thank you.

Rebecca: Thanks, Linda.

Linda: Bye bye.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.