351. Extending Kindness

Beginning faculty often receive warnings that lead to antagonistic relationships with their students. In this episode, Cate Denial joins us to discuss how a pedagogy of kindness can build productive learning environments for all students.

Cate is the Bright Distinguished Professor of American History and Director of the Bright Institute at Knox College. She is the winner of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award and sits on the board of Commonplace: A Journal of Early American Life. She is also the author of A Pedagogy of Kindness, one of the first publications in the new Oklahoma University Press series on teaching and learning, edited by Jim Lang and Michelle Miller.

Show Notes

Transcript

John: Beginning faculty often receive warnings that lead to antagonistic relationships with their students. In this episode, we examine how a pedagogy of kindness can build productive learning environments for all students.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by

John: , an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Cate Denial. Cate is the Bright Distinguished Professor of American History and Director of the Bright Institute at Knox College. She is the winner of the American Historical Association’s 2018 Eugene Asher Distinguished Teaching Award and sits on the board of Commonplace: A Journal of Early American Life. She is also the author of A Pedagogy of Kindness, one of the first publications in the new Oklahoma University Press series on teaching and learning, edited by Jim Lang and Michelle Miller. Welcome back, Cate.

Cate: Thank you.

John: Our teas today are:… Cate, are you drinking tea?

Cate: I am. I am drinking Harney and Sons Decaf Earl Grey.

John: …very nice.

Rebecca: It’s a great choice. I have a green tea today.

John: A what tea?

Rebecca: I don’t know it’s just green tea. [LAUGHTER] Sorry.

John: And I have a peppermint spearmint blend today.

Rebecca: Sounds nice and calming. So we’ve invited you here, Cate, to talk about the Pedagogy of Kindness. I think we last talked to you in October of 2021, when the book project was just getting started. And our discussion focused on a “Pedagogy of Kindness,” an essay on the topic that you posted in I think, August of 2019. This essay was such a great resource that helped so many people and I know, our community here was really interested in the subject. Can you talk a little bit about how this essay evolved into a book?

Cate: So, as I wrote the essay, there was so much I wanted to say, and I had to sort of just express it in the most concise form that I could. After I was done, I was sort of pondering the other things that I wish I’d had the space to be able to include. And then Josh Eyler, who is now at the University of Mississippi, messaged me and said, “Hey, I think this needs to be a book.” And I was like “Me too.” [LAUGHTER] So I started thinking of what I would do to make it a longer treatment. And I wrote up a proposal, sent that off, and started writing, pretty much straight away.

John: In this book, you argue for a teaching approach based on kindness. You note that many people, though, confuse “kindness” with “being nice.” How do you define “kindness?” And how is it different from “being nice.”

Cate: So, being nice is a lot of what we do in academia, nice lies, nice will do anything to just get along, it will avoid conflict, and it will just try and paper over cracks in our institutions, whereas kindness is fundamentally honest. So it means you’re going to have some really tough conversations with your colleagues and with your students. You’re not going to mislead them about how they’re doing in your class. And kindness is really about accountability, it’s about positionality, and it’s about responsibility. And those are all things that I think niceness just elides.

Rebecca: I know that I found myself saying many times after reading your essay: “Telling the truth here or letting a student know what their real situation is, would be the kind of thing to do. [LAUGHTER]

Cate: Exactly,exactly. And I often have people, as I go around the country and talk to other faculty, ask me: “Exactly how do you have these conversations, when the student comes to you and says, ‘this terrible thing has happened, and please just excuse this paper,’ or ‘don’t include this in my grade’…and you have to, especially if you are in a professional school, for example, or a medical school or something like that.” And that’s when that distinction between niceness and kindness really comes into play. Because niceness is just about getting along. And kindness is like, “Well, let’s have a real conversation about what’s going on here. Why these expectations are, what they are, whether it’s possible for us to make some kind of adjustment here, or whether this is a moment where we really have to talk about what your future is in this course, or in this program,” things like that.

John: In the introduction to the book, you note that this is not quite how you started teaching, that you started teaching, like many of us, with a somewhat different focus in terms of interactions with students. Could you tell us a little bit about your perspective when you first entered teaching?

Cate: So when I first entered teaching, I was brand new to teaching. I had got my degree in England one month before I stepped into a classroom in the United States. I did not know the United States educational system well at all. I felt like a complete fraud, and the limited training I had had at my graduate program really encouraged me to think of my students as antagonists, to assume they were going to cheat, they were going to try and get one over on me, that they wouldn’t do the reading, that they were going to try and negotiate for a grade that was not what they deserved to get, things like that. And walking into a classroom like that prepares you for battle [LAUGHTER] at every single moment, which is a terrible way to walk into a new relationship of any kind. It’s especially demoralizing and damaging, I think, when you are walking into a classroom, so I had to unlearn the business of thinking that my students were up to no good. I had to unlearn the business of thinking that my students were fundamentally not interested in their own education.

Rebecca: You mentioned your four day visit to a digital pedagogy lab, and how that impacted your teaching. Can you elaborate a little bit on that and share that story?

Cate: Yes. So I went to the Digital Pedagogy Lab Institute in 2017, which was in Fredericksburg, Virginia. It was the most incredible experience. I went because I had the feeling that I was lagging behind in bringing digital tools into my classroom. And I thought this would be where I would learn those things. And I certainly had that opportunity. But the first thing that I was asked to do in the intro track, was to tear apart everything about my teaching, in a really good way. I was in a track that was led by Chris Friend and Sean Michael Morris. And they asked us to reflect upon everything about our pedagogy. Why were we doing things in the way that we were doing them? What was the rationale that we had for the choices we were making, and the way we addressed our students and the kinds of things that we have happen in our classrooms. Everything about that institute was predicated on kindness. And it’s not that someone actually said at some point, “Why not be kind?” But that was the subtext from everything from… it was the first Institute I think I ever went to where there were pronoun pins that people could take when they registered. There was really generous swag, there was this wonderful ethos of sharing, everybody rolled up their sleeves. and really worked hard at turning everybody’s teaching materials into something even better than what they arrived with. There was really no hierarchy. It was just a wonderful community experience of trying to get better at what we do.

Rebecca: Sounds like such a great experience, and an opportunity to really critically reflect on your teaching, which we don’t always take the time to do.

Cate: I didn’t even know I needed to do it. [LAUGHTER] So by the time I went in 2017, I was more than 20 years into my teaching career. And I thought I was pretty great at teaching at that point. But what the digital pedagogy lab made me realize was that no matter what my goal was, as a teacher, all of the documents in my course, my syllabi, my assignment sheets, the way I handled introductions, all kinds of little things, really told a different story. And they told that story of antagonism that I had been socialized into as an early instructor. And I think that is something we are socialized into globally in higher ed.

John: One of the things you talk about is finding out a little bit more about your students. Because when many of us start teaching, we tend to assume that our students are similar to us, they have similar backgrounds, they’ve been prepared in ways that were similar to ours, maybe they weren’t quite as good as students as we were. But many people start off by assuming that students will be just like them. And yet you note in your introductory chapter that as many as 71% of undergraduates in the United States are non-traditional, under a teaching approach that focuses on kindness, how might faculty begin to address the diverse identities and needs of our students?

Cate: So, I think the first thing that we need to do is take stock of who our students are. And so we need to think about the demographics on our campus, that information is usually pretty easily available. As I travel around the country and visit different campuses, I always pause and look up on their websites, what is their diversity rate? And how many low-income students are there? How many people get Pell Grants? How many people are first-gen? How many people are returning after taking some time off? Or maybe have a GED? There’s all kinds of ways in which our students are not that sort of stereotypical 18 to 22 year old, straight white student who is perhaps in a sorority or fraternity or does athletics or anything that sort of gets caught up in the stereotype of who students might be. I’m really interested in what happens behind that. Even if a student walking into my classroom is, for example, a straight white student who is 18 to 22 years old, or maybe is an athlete, maybe is in a sorority. That doesn’t mean that those things tell me who they really are. They don’t tell me what is going on in their home life. Do they have kinship obligations and responsibilities? What is happening with the way that they’re funding their education, all kinds of different things. So I think pausing at the beginning of the term and really getting to know who our students are, is vital if we’re going to meet them where they’re at in our classrooms. So there’s lots of ways of doing that. There are ways of sending out surveys to our students, either before the semester begins or right as it begins. There are ways to have conversations in class to make those introductions a little bit more substantial than just, “Hi, my name is….. And my major is….. , and I’m taking this class, because….. So trying to gather that information, and also reflecting on how my identities are showing up as I walk into a classroom. I think those are really important parts of what we do.

Rebecca: What are some of the ways that adopting a pedagogy of kindness changes the ways that instructors and students interact?

Cate: I think it is all about undermining that presumption of antagonism. If we do not assume that our students are out to get us, But rather, that our students, in the vast majority of cases, are actually really invested in being there, and are perhaps a little bit scared of us, that they are a little bit uncertain, a little bit unsure, maybe they are first years, or this is their first class in a certain discipline, or they’ve been out of school for a while. There’s all kinds of ways that they might be very, very nervous coming into our classrooms. And so I think, taking the time to think about that, and address that, and make space for that, so that our students are not some sort of terrible, antagonistic, cruel set of people who really don’t want us to succeed or for themselves to succeed. That’s just not based in reality.

John: And I would think it would be a much more enjoyable environment, both for students and for the instructor, when there’s that trust between students and faculty.

Cate: Yeah, I actually have a blog post that I wrote, I think back in 2016, called the first-day jitters, that was about how nervous I would get before I would meet my students for the first time every semester. And once I changed my outlook and stopped expecting antagonism, I stopped getting nervous. I walked into the classroom and was excited about what was going to be co-created in that space. And it made an enormous difference to me and to my students. Certainly, I saw it in the ways that they interacted with one another and with me in our classroom spaces, but it changed everything for me about showing up as a teacher.

Rebecca: One of the things that you’re pointing to or hinting at is the idea that the pedagogy of kindness also applies to being kind to yourself. Can you talk a little bit about ways that instructors can be more kind to themselves?

Cate: Yeah, I think the number one way in which we can all be really kind to ourselves is to have impeccable boundaries. [LAUGHTER] I think it’s really important that we not prostrate ourselves, not give beyond what we are capable of giving, and have a really good understanding of what our capabilities are. So for example, if I am someone who is stringing together three jobs to make a livable income, if I have a family, if I have caregiving responsibilities, if I have a really long commute, all of those things are in the mix. And I cannot then be someone who 24/7 is simply available to my students. I don’t think that’s healthy, even if you don’t have any of those other responsibilities or considerations. I think that we all need time to have downtime, to have rest, to have play to, have pleasure. But I think that we need to be able to say, “I am available by email between these hours, you can expect an answer from me within a certain period of time.” Communicate those things incredibly clearly to our students. And that means that we’re also modeling for them that it is okay to take time for yourself and to have good boundaries.

John: In the second chapter of your book, you focus on the syllabus, which is often one of the first forms of communication between the instructor and students. Quite often the syllabi look more like legal documents than they do a welcoming introduction to the course. What are some of your suggestions on how to demonstrate kindness in your syllabus?

Cate: So an exercise that I was asked to do at the Digital Pedagogy Lab was to read my syllabus with a critical eye and ask myself who I was addressing when I wrote it. And what I found was that I was addressing someone who I assumed was going to screw up somehow. I assumed that they were going to disappoint me they were going to let me down, that they were going to try and get away with something… that old antagonism rearing its head. So then I had to stop and go, “That’s not who I think my students are, so why is that reflected in the language I’m using in my syllabus?” So then I stopped and I rewrote. And I kept in mind a student who is very real, I pick students to have actual students in my mind when I’m writing my syllabi, or when I’m revising them. And in addressing those students, I was able to make my language welcoming and warm. It doesn’t mean that it’s soft, it doesn’t mean that I don’t have boundaries and that I don’t have expectations. It simply means that I’m talking to them as I would if we were in person. That’s one thing that I learned to do. Another is that I learned that for many students, syllabi are impenetrable walls of text. And I’m sure all of us, in our time, if we haven’t written that syllabus, have seen that syllabus, maybe when we too were students. Those are really hard for students who have things like ADHD, for example, to be able to parse, to find the important information. They don’t look welcoming. They look like legal documents. And so I’ve changed my syllabi, for example, to have really colorful welcoming headers. I make a point of always saying “welcome to the class,” as the first thing that you see on the syllabus. I make a point of emphasizing relationship early and saying, “This is how you can get hold of me. Here are what my student hours are and what student hours mean.” And only then do I get into the stuff like, here’s what we’re going to read and here’s the assignments we’re going to do. It’s all about establishing warmth and a relationship from the get go.

Rebecca: How do you balance that approach with some institutional policy or other things that you maybe have to include in a syllabus that don’t have that same tone and tenor?

Cate: Yeah, that’s a really big consideration for a lot of people. There’s a lot of boilerplate language that has to go into many syllabi. So there, I borrow from Remi Kalir, who has students annotate the syllabus, and I’ve been doing this for several years. Now, my students get the syllabus at the end of the first class session, and they take it away for homework and they annotate it. I give them a really simple definition of what annotation is. And then in our second class period, we come back and they get to tell us what their questions were, what their comments were, what seemed unusual to them, what really stuck out and we have a conversation about all of that. And if you have a bigger class than I do, this is the sort of thing that you can have small groups do or you can have parents do. And then you can have people report out some of the things that they thought were most pressing. What that does is it returns to students some agency, and they can comment on the language shift, the tonal shift between the things you wrote and the things that you have to include, and you can have open conversations about “So why do you think we have to include this stuff? And what are ways that we could say this differently?” or “What is the way that you understand this best,?” Having that conversation is really important in allowing students to talk back to that document. And to make it clear that it’s a human creation, that it’s not just spit out by a machine somewhere, that there are people behind it, and that we are collectively always polishing and revising our syllabi in light of these relationships that we have on our campuses.

Rebecca: In smaller classes, I’ve done the annotation in a single document. And it’s been really interesting to have students comment on each other’s comments in a digital format, too. And it really brings some interesting things to light.

Cate: That’s a great way of doing it. I tend to have my students do it individually, just because they don’t know each other very well yet. And I found that my students tend to respond better to individual documents, and then a group discussion, but it works differently with every single instructor and every single set of students. So there are multiple ways that people could do this kind of annotation.

John: You did mention that this syllabus is not something spit out by a computer, but last week, we had a workshop where one of the things I suggested was that faculty may want to submit their syllabus to one of the AI tools out there and have it checked for inclusive language or to rewrite it in a more inclusive manner. And I think a lot of people found that, in that case, the computer did a better job than they did [LAUGHTER] in terms of creating that inclusive syllabus. But, at the other extreme, one of the things I’ve often done and I think you mentioned this in your book is in smaller classes, co-create the syllabus with students at the very start of the semester.

Cate: Yeah, I teach a course on history pedagogy. And there I give my students blank sheets of paper and say “Here’s your syllabus,” [LAUGHTER] and then they get to start brainstorming the things that they should put in syllabuses. They’re experts at syllabi, they read them all the time, they see many more than most of us do. And so they brainstorm what they think it should include. And then I put them in small groups. And together, they start to talk about, “Oh, that was a great idea you came up with, I didn’t think of that. Let’s put that in our syllabus.” Then we write their syllabi up on huge post-it notes that we put on the wall. And then everybody gets little post-it notes. And they get to walk around, and I ask them to do what I was asked to do at the Digital Pedagogy Lab, which is to say, “Who is the student that these authors are imagining as they write the syllabus?” Write it on your little post-it note, put your reactions on the wall. And what they very quickly learn is that they too have absorbed the idea that students are antagonists, students can’t be trusted, students are going to mess up somehow. And so then, as a class, we sit down together, and we say, “Okay, we don’t want to do that. We don’t want to make those assumptions. So how are we going to write a document that accurately sets out expectations, that also supports and that communicates trust?”

Rebecca: In chapter three, you focus a bit on assessment. And many faculty note that when students do less well than maybe they thought they were going to or anticipated on a graded activity, they might become disengaged or lose interest or disappear. Can you talk a little bit about ways that we can make our assessments more equitable, and to make expectations more transparent to our students?

Cate: There are so many ways that we can do that. Including students in the process at all points, I think, is a way that we can never really go wrong. But I think that one of the most important ways we can approach assessment is to think about, are we being inclusive? are we giving our students the tools they need to be able to show us what they know?…because so often, what we’re actually grading or assessing or giving feedback on is, how they write about what they know, or how they make an equation about what they know, we’re not actually just finding out what they know, what they have learned. So giving them multiple ways to express what they know, is a great way of leveling the playing field of allowing students who maybe don’t excel at writing to still be able to show “Hey, I have been thinking, I’ve been turning this over and over, I have some new appreciation for whatever it is art, or music, or fishbones, or an equation, or set of chemical compounds.” So in my classes, it’s traditional for history to be assessed through writing papers. And I certainly do have my students do written work, I don’t get rid of it altogether. But I also make an assignment where my students get to create projects. And they can do anything they want to, except write a paper. So they do things like make quilts, and food, and board games, and they knit and make pottery. I’ve had people make raps and sea shanties. I’ve had people dress up as pirates and come talk to us about a day in their life. All of these things are based on historical research. All of these things have a reflective component at the end. But for a student who really struggles with writing, for example, this gives them a way of being able to show me they have been learning that term, that they have done research, that they’ve really engaged with the class material in a way that doesn’t limit them in any way.

John: In chapter 4, you address kindness in the classroom. Could you share a couple of ways in which faculty can display kindness in their classroom interactions?

Cate: I think one of the most basic and important ways that we show kindness in the classroom is we learn who our students are. And that means names. And that is increasingly challenging. It depends on how many, if you’ve got 300 students in your classroom, how do you learn their names? I tend to have smaller classes. So it is really doable for me to learn their names, but I still rely on tools that helped me do that. So I bring in name tags, and for the first several sessions, my students will write their name on their name tag, and if they feel comfortable, they’ll put their pronouns on there too. I know that Vigi Sathy and Kelly Hogan talk about having name tents in their big lecture classes. And those name tents are also color coded. So then that helps them put people into small groups. People can turn those name tents on their end when they want to indicate they want to participate in conversation. They’re wonderful, wonderful tools. So there’s lots of resources out there that sort of suggests there are many, many ways to make sure that we’re getting to know our students, but I think that’s a really basic thing we do that says, “I know who you are. I know when you’re here, I really appreciate you being here and being engaged in a classroom activity.” I also think that it’s important to teach our students how to be in that space together. I think this is really important coming out of the height of the pandemic. We’re not done yet. but we are coming out of those most acute years. And certainly I have seen among my students on campus, that there was a period of time where it was odd for them to interact with other people because they hadn’t had practice. They had been in lockdown. They had been at home. They had been doing school online. And so we really had to sort of think about how do we interact? How do we talk to one another? How do we show that we respect one another? When something goes wrong, how do we show that we are accountable and responsible? How do we apologize, and mean, it. These are all things that we need to teach, as instructors to make sure that our classrooms are kind spaces in which learning can really happen.

John: Is there anything else you’d like to share with our listeners that we haven’t yet discussed?

Cate: One of the things that comes up in the conclusion to my book is that it’s very easy to feel tossed around in higher education, like we’re in tiny little boats on a very large and angry ocean.[LAUGHTER] And certainly, there are structural problems that need our attention in higher ed. But we’re not powerless. And we get to make changes in the spheres that we control. And we get to approach our students, as fellow human beings who matter, we get to treat ourselves as people who matter. And kindness is the fabric of that, it’s what weaves everything together. It is the material from which we build inclusive spaces, and spaces where people’s words and ideas are valued. And without kindness, we end up with places that are disrespectful, we end up with places where people do not feel like someone’s listening to them. So I have the metaphor, that little boat being tossed around in my conclusion, but ultimately say, if we can just be kind to one another, then we can sort of temper that storm.

Rebecca: It’s a great image to end on, for sure in your book. And it’s such a great metaphor for really thinking through what sometimes it really feels like to be in higher ed, not only with our students, but with other colleagues. There’s lots of ways to bring this kindness work into other spaces within higher ed outside of the classroom as well.

Cate: Yeah, I agree. And one of the projects I’ve been involved with over the last couple of years is something called Care in the Academy, which is about showing kindness and care to each other. So focused on faculty, focused on staff, focused on those relationships, and how we show one another that we matter to each other.

Rebecca: Can you share some examples about what that might look like in spaces outside of the classroom?

Cate: Yeah, so some really concrete things that we recommended after 18 months of really thinking critically about this issue were things like extended bereavement leave. Places tend to give us three, maybe five days and they’re immediate. And the idea is that whatever mourning or immediate grieving you need to do will happen in like a week. And that’s just not practical or realistic, or how humans work. And in many cultures, it’s also not reflective of the way that people process death. So we talked about having maybe another three to five days that you can use for a year after someone passes and you can mark an anniversary, or maybe there is a ceremony that you need to attend. Or maybe there is a ritual that you need to perform, and you take time away to be able to do it. It’s a very small thing, but it makes such a difference in sort of saying we respect what a difficult time this is for you. Trying to improve reimbursement culture, there are so many people who are stringing things along on their credit cards, basically making loans to their institutions, and trying to speed up the process by which they are repaid as a basic form of respect, I think is really, really important to the way that we all function around one another. To not assume that people are sitting on vast mountains of cash that they can just make available to do things like go to conferences or to travel to another town or whatever the thing is. I think that’s also really important.

John: That’s especially true for junior faculty, for people who are contingent faculty and who are struggling a bit because this occupation does not pay all that well. It gets a lot easier once your kids are out of the house and you’re past those stages, but it’s a struggle for a very large share of faculty.

Cate: Especially given student loans being what they are. This lasts a very long time.

Rebecca: It’s something that sometimes faculty have in common with students and we don’t always recognize or acknowledge that.

Cate: Yeah, I agree. And I think our students don’t often look at us and think, “Oh, wow, they’re still paying off their student loans.” [LAUGHTER] They tend to see us as something other than precarious. Whether we are in a contingent position or not, very often our financial lives are in some kind of precarious situation, just because the reality of funding higher education careers is what it is right now.

Rebecca: So we always wrap up by asking: “What’s next?”

Cate: What’s next is that I am going to go read a book about biomimicry. And biomimicry is looking to the natural world for patterns that all organisms share, and taking wisdom from those patterns. So it helps us in terms of ecology. But I just heard a podcast that was applying this to the way that we organize our institutions, and thinking about the way that organisms heal after fire or clear cutting or a flood, for example, the different jobs that different organisms have in the recovery process, and thinking about how we make really resilient institutions and organizations by looking to the natural world. So I know very little about it. I’ve listened to that one podcast. So I am going to go read a book that is all about that topic.

Rebecca: That sounds really fun.

Cate: Yeah, I think so.

John: And I think we’ve all needed a bit of resilience over the last few years with the challenges we’ve all faced. So it sounds like an interesting way of addressing that issue.

Cate: Yeah, I’m hoping I will get some fresh ideas.

Rebecca: Reading stuff that’s unfamiliar is always a great way to reinvigorate our own work.

Cate: Yeah, and I have not really considered biology since I was 16 years old and took my GCSEs. So this will be a real treat and a delight to exercise part of my brain that hasn’t been exercised in a while.

John: Well, thank you for joining us. It’s great talking to you. And I really enjoyed reading through your book. It grew from this very nice, useful essay that many of us were reading through during the pandemic, to a much more elaborate discussion of how to be an effective teacher in all aspects of teaching.

Rebecca: Such a needed robust resource.

Cate: Thank you so much. It was such a pleasure to be with you today and have this conversation. And I really appreciate all your kind words about the book.

Rebecca: We look forward to talking to you in the future, hopefully the near future.

Cate: Thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

350. Nudging, not Judging

During the pandemic, faculty participation in professional development activities expanded dramatically. Faculty involvement, though, has been gradually returning to pre-pandemic levels. In this episode, Sarah Rose Cavanagh joins us to discuss strategies for bringing more faculty into discussions of teaching and learning.

Sarah is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and also is a regular contributor to the Chronicle and many other publications. Sarah often serves as a keynote speaker and we were very fortunate to have Sarah join us for a keynote address at our Academic Affairs Retreat in Oswego last August.

Show Notes

Transcript

John: During the pandemic, faculty participation in professional development activities expanded dramatically. Faculty involvement, though, has been gradually returning to pre-pandemic levels. In this episode, we explore strategies for bringing more faculty into discussions of teaching and learning.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Sarah Rose Cavanagh. Sarah is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and also is a regular contributor to the Chronicle and many other publications. Sarah often serves as a keynote speaker and we were very fortunate to have Sarah join us for a keynote address at our Academic Affairs Retreat. Welcome back, Sarah.

Sarah: Thanks for having me.

Rebecca: Today’s teas are:… Sarah, are you drinking tea,

Sarah: I thought about trying to blow your mind and saying I’m drinking some Bergamot green thing. But no, I just finished my last coffee for the day. And now I’m on to water. [LAUGHTER]

Rebecca: You’re so consistent.

John: My tea today is a black raspberry green tea.

Rebecca: And I have an Awake tea, which clearly I need. [LAUGHTER]

John: We’ve invited you here today to discuss your May 7, 2024 article in The Chronicle, entitled, “After a Tough Year for Classroom Innovation, It’s Time for a Reset.” And I think this is something that’s going to resonate with a lot of educational developers and many faculty. You begin the article by noting that there was a sudden increase in attention to student-centered teaching in response to the pandemic, but has that shift been as persistent as many of us might have hoped?

Sarah: Oh, I think yes and no. I think that a lot of people started focusing more on their teaching. In those pandemic years, we all saw in our teaching and learning centers, higher attendance at events and a lot of engagement, both in these larger events, and then faculty coming to us for consultations. And one hopes, as educators, that learning doesn’t just go poof, that the effect of all of that attendance and thinking and consulting is still impacting people in their classrooms. But I think that there definitely has been a trend on my own campus, and most campuses that I talk to, about declines in attendance at those same sorts of workshops and talks and consultations. And there also, I think, has been a bit of a pendulum shift in some of the topics such as flexible deadlines, where I think we saw a lot of people move in one direction, and then are starting to see a little bit of a shift back. And so hopefully, we still have overall nudged the average over towards student-centered teaching and that hasn’t changed. That’s my hope. But I definitely see a little less intensity and interest lately.

Rebecca: We definitely have seen similar declines in professional development on our campus as well. And sentiments like, “I’m just really tired,” or “I’m done” [LAUGHTER] which you highlighted in your article, and one of our most popular workshops of recent years has been related to burnout. Can you talk a little bit about ways that we can re-engage our faculty or why burnout is such a popular topic? [LAUGHTER]

Sarah: Right? Well, I think that if we’re thinking about kind of a shift away from student- centered teaching, or even a backlash, as we’ll talk about in a little bit, that burnout is one of the many causes of that. I think that faculty are just tired, as you said, and that sentiment of “I’m just done,” like, I’ve heard those words multiple times from multiple different people. And a lot of student-centered teaching, and certainly attending professional development workshops takes time, and there’s just no time. And so there’s the burnout that’s related to the emotions, and the trauma of the pandemic, and just workload being completely ridiculous. But then there’s just also a paucity of time. And I think that part of that is this whole Zoom… we’re here, not on Zoom, but we’re here virtually, and those virtual options open up so many possibilities. But I feel like we took all of our pandemic solutions with the Zoom sessions, and all of that, and we brought them into the future. But then we brought back all the in-person stuff. [LAUGHTER] And so there’s zero time for anyone. And so I think that the burnout is so strong, and that’s why there’s such attention to that. And that is one of the most popular topics right now.

Rebecca: Feeling over scheduled definitely limits the ability to feel creative, or to dream or to think about new things. So if people are used to kind of being in that state, or historically have been in that state, and they’re just feeling like they can’t get to that state. I can imagine that’s why people say they’re done is because it’s hard to come up with new ideas or to put the energy into that kind of thinking.

Sarah: Absolutely. And I think also students are equally, at least equally, burnt out. [LAUGHTER] And so I think another thing that’s happening is faculty put all this time into student-centered teaching, but then it’s not working. They’re not seeing more engagement or better learning. And so you’re investing all this time, but then the students are so burnt out that it doesn’t look like it’s being effective, and that’s frustrating, and I think that that’s also part of the feeling done, like, if you’re just throwing everything you have at bettering your teaching, but then still feeling frustrated by the outcome, then that is demoralizing.

Rebecca: It’s interesting that the way you put it instead of like getting re energized… [LAUGHTER] like, depleted.

John: And we never observe the counterfactual, we don’t see what would have happened, had we not been focusing more on student-centered teaching after some really tough times. So, why do you think faculty are pushing back a bit more against student-centered teaching approaches, because we had a lot of buy-in on that during the early stages of the pandemic, and for a couple of years after that, but we’re seeing a little bit more pushback.

Sarah: Yeah, well, I have a long answer for this one, because I am a psychologist and so I think everything is highly multi-causal. And one major factor is the level of burnout, both on the faculty side and the student side. And so we don’t have to go back to that, but I just want to highlight that I think that that’s part of it, is this burnout, and I love Rebecca’s depiction of like how impossible it is to feel creative and do new things too when you’re feeling exhausted. And so I think that’s a big part of it. On my endless commute yesterday, [LAUGHTER] there was a Red Sox game (I work in the Fenway), I listened to your interview with Josh Eyler, which was excellent, of course, and I think it probably predated the article, but you were talking about just this topic of sort of a backlash, specifically against alternative grading schemes as a student-centered practice. And Josh said, that I thought was very insightful, as everything Josh says is, that he had this analogy of a rubber band, and that there’s just this effect. And I think of it as a pendulum as well, which I already referenced, that there’s just this natural effect when you move in one direction to kind of snap back a little bit. And he saw that as part of the cause of this. And I think that that is true. I think it’s sad, but we love pendulums as human beings. I’m always like let’s just settle into moderation, [LAUGHTER] but we love swinging from one extreme to the other. So I think part of it might just be a regression to the mean sort of effect. But I think there’s a lot of other factors. I think that one is, if you drill down into the recommendations of so many different theoretical approaches to good teaching, you come up with similar recommendations, whether you’re focusing on inclusive teaching or accessible, teaching, engaging teaching, all these things, they all drill down to these recommendations of being warm and welcoming, having a promising tone, seeing students as people, giving them autonomy and giving them choice, all these things. And I think that some faculty feel, after attending several years of the pandemic and all these different intros and frames, but then the same recommendations feel like they got it. [LAUGHTER] And that might be part of it. The final two causes of the backlash are a little more delicate. I think one is the student reactions. And we talked about this a little bit, that students themselves are so burnt out and so faculty aren’t seeing great results when they experiment with student-centered teaching. But I think that there’s also a rise in student incivilities. Beth McMurtrie had a great report in The Chronicle, about a year back on the rise of student incivilities. Another term that people use is reactance, when they’re pushing back against the instructor. I can tell you a brief anecdote. I was on a different campus talking to a faculty member. And she had been using alternative grading practices, it was kind of a standards-based approach where students had multiple attempts at achieving a standard, but they’re passing an assignment and would either be competent or not yet competent. And she was teaching in a male -dominated field. And this student of hers who was male, he got a “not yet competent,” and threw the paper back in her face and said, “You don’t know what you’re doing.” And so I think that that too, when you’re putting all this time and effort into student-centered practices, and then having reactance and incivilities in reaction, that’s also demoralizing. And then finally, just to get myself in trouble, [LAUGHTER] I think that we in the teaching and learning fields, bears some responsibility. I think that especially toward the end of the heyday of teaching and learning years on Twitter, things got a little bit ugly. There was some shaming of faculty pile-ons, and some of the pile-ons were of people engaging in student shaming practices and other things that are ugly on their own. But some of it was also just venting stresses of teaching, and then getting piled on. And I saw this really clearly because I was active on Twitter in the teaching and learning fields. And then because I was more of a research focused faculty for most of my career and all my social contacts in person are more like that, I’ve had all my faculty friends on Facebook, and they would post just some things. They all very care deeply about their teaching and their students, but they would post things and I was like, “Oh my God, if you had posted that on Twitter, [LAUGHTER] you would be dead in the water.” So I think that we, as we talk to each other, and we convince each other, and we get more and more excited about change, and about some innovative practices, that sometimes we get a little out of touch with the majority of faculty who are teaching on the ground, and their constraints and practices.

Rebecca: We spent a lot of time talking about students as humans, but teachers are also human, I think, [LAUGHTER] and I think we need to be, and you’ve mentioned this before, too, is just making sure that we’re treating everybody as humans, and that everybody has constraints, and everybody has different pressures. And in your article, you suggest that the time seems ripe for a complementary revolution, one that recognizes and respects the autonomy and bandwidth of teachers. Can you talk about what this counter revolution might look like?

Sarah: Sure. And this is easy for me to say, because in my position I have no control over budgets, [LAUGHTER] or faculty workloads. But I think that primarily what we need is we need a revolution in how we treat faculty. Faculty have been through the pandemic, I think the coming demographic cliff and all of these things that administrators push for faculty to be more and more responsive to students, more and more student-centered, to take on more and more things. At my last institution, they had us writing letters, personal letters to individual possible students to try to encourage them to put down deposits. All of this emotional labor time, regular labor. And so I think faculty in response need more time and money. And I think we’ve just piled on and increased the workload and increased the workload and expanded the scope of the work exponentially through these pandemic and post-pandemic years. And I think that faculty need course releases, I think they need stipends for some of this work. And I think that we need to put a lot more resources toward our faculty. I think it also means a lot more full-time lines with security as well. And those are going in the opposite direction. I think that the other thing that we need is hands off our classrooms. [LAUGHTER] I think that we need to respect faculty autonomy and faculty choice. And so many of us get into this work, because we want that autonomy, we want that choice. We’re willing to work harder for a lot less money than if we went elsewhere. But it’s because we want to be self determined, and do the things that we want to do. And I think that we need to respect that.

John: One of the things you’re suggesting in the article is that there’s no single approach to student-centered teaching that works for all faculty or for all students. Could you give some examples of approaches that work for some faculty in their context that might not work as well for other faculty, given their context?

Sarah: Absolutely. There’s so many… class size is one, you know, are you teaching 25 students? Are you teaching 750 students? …very different strategies are going to be effective there. Courseload… are you teaching two courses a semester, or you’re teaching five courses a semester, that is going to affect the choices that you can make. Precarity of position… both pre-tenure and post-tenure, but also: Are you full time? Are you contingent? Are you on a semester to semester? Contract… identity, these characteristics of the instructor. We had Chavella Pittman come out and talk to our faculty at Simmons University last semester. She was fabulous. She talked a lot about student incivilities, and how many more faculty of color and other minoritized faculty receive of those student incivilities and what majority faculty can do differently in terms of their policies to help support their minoritized faculty. Your teaching philosophy, I think, is important. And again, autonomy is important. And a little bit your teaching talents. For instance, I’m not funny. [LAUGHTER] Some people are really funny. And that is like being humorous in front of the classroom is a really engaging strategy, but it’s not open to all of us. And some people are gorgeous lectures and actually, lecture might work better in their class than someone who’s not. And there’s just so many contexts, there’s so many constraints that are going to influence this vast buffet of teaching choices.

Rebecca: Sometimes when we’re thinking about offering students choice, that also means there’s an impact on faculty too. And so you’re kind of underscoring a faculty choice and some of the individual faculty needs where they might be balancing other things may have certain kinds of time constraints, physical limitations, there’s all kinds of things that could be at stake. How do we help folks understand the need to balance student needs with also faculty needs, and to make those kinds of decisions?

Sarah: It’s so tricky. I often think of all of teaching as kind of a dance, and there’s an art to it. And there’s a little bit of an intuition to it. I’m mad at myself, because I meant to look this up before this talk, but I will send you a link to something later on. Brian Dewsbury and some other folks put together an inclusive teaching massive online course through HHMI. And I lead faculty through a learning community using it. And they were discussing, and I’ll send you the link and the person’s name after. But it was an approach to teaching that, instead of being learner centered, it’s learning centered. And this balances out this competing tension between faculty needs and student needs, but also between, and I think we’re going to talk a little bit about this in a bit, students to student needs, you know, variations in the needs of students. And instead of centering the learner, centering the learning, and thinking about what is going to be most effective for the goal of this course, which is learning, and what can I do in terms of my teaching choices, and the structure of the course and the types of assessments and things like that, that centers the learning, which is going to center the learner as well, because it’s their learning that will happen. But I liked that subtle shift in how we think about this. And I think that might be one approach to approaching this dance with balance.

John: And that brings me back to when I first started working at the teaching center. The year before that an advisory committee put together a draft statement of the purpose of the teaching center, and they went with learning centered. We’ve still tried to maintain a focus on learning, because that’s kind of what we’re here for. And that’s why students are here. So I think that focus is really good.

Sarah: Oh good. [LAUGHTER]

John: But going back to that issue about different techniques working for different faculty, isn’t the same also true for students? That the flexibility that many faculty provided to a greater extent over the years immediately following the pandemic might work really well for many students, but might that provide a loss of structure which is really important for many of our students?

Sarah: Absolutely. A question I often get when I give talks about emotions in the classroom is about my recommendations for a lot of warm-up activities and small-group discussions, and getting students interacting with each other to maximize the sense of belongingness and community, which we know is important for learning and especially for marginalized students. But I often get the question of what about socially anxious students? What about introverted students? What about neurodivergent students? A lot of these students might really struggle having so many social engagements during a class period. And I don’t know that my answer is very helpful, [LAUGHTER] because I always try in my own teaching to have varied methods. So it’s not always social, that there are some classes that are much more independent, there are options to write in small books and things like that. And in the piece, I referenced Sarah Silverman’s work on access friction, and she didn’t coin the term she gives credit to Areley McNeney, and others, but about this friction that arises when one student’s accessibility needs are different, or even opposing in tension with another student’s. I think variation is good, you know, varying things around and flexibility and talking to your students, using who’s in class surveys to find out about student needs, always respecting students’ accommodations, of course, but having that conversation with students and drawing them in. And one thing that I really tried to do in my own teaching, is to be transparent about why I’m doing a certain thing. So I often start my classes with an explanation for why there’s going to be so many community building sorts of activities and how it is the research suggests that that’s important for their learning. So we think that kind of transparency, varying strategies, not doing the same thing all of the time, and then just acknowledging that certain teaching strategies are going to be more effective for some students versus others. And so let’s not just blast one teaching strategy the entire semester.

Rebecca: I think one thing that often happens is the use of the term universal design, not necessarily universal design for learning but this idea of universal design implies that one size can fit all, but usually universal design, if we’re really following those principles is that there’s variability and is that the space or the technique or the situation can adapt to the person and vice versa, rather than necessarily having one solution that works for everyone. I mean, it’s part of it, but there’s usually this flexibility piece that’s key, but flexible doesn’t have to mean so much extra work for a faculty either. There’s different ways to have flexibility.

Sarah: Yeah, multiple points of entry for students always is great.

John: When I first started teaching, I was told by the department chair at the time, who’s long since retired, don’t waste too much time on teaching, spend your time on research and other things that are essentially the currency of higher ed. We’ve moved quite a ways from the time where many faculty are getting that sort of advice, and those of us in educational development often become really passionate about encouraging student-centered teaching approaches, and we generally provide some criticisms of high-stakes exams, the sole use of lecture, multiple choice tests, and so forth. Those were pretty much considered to be standard practice not too long ago, and are still very common in many disciplines at least. You mentioned Twitter, many people would consider some of those techniques to be sort of like malpractice, where they’re actively doing harm to students and learning. Have we perhaps gone a little bit too far in pushing for changes in instructional practice, in the direction of a one-size-fits-all approach that perhaps we should back away from somewhat to bring more people into the conversation?

Sarah: Sure. So multiple layers there. I think one thing is, this is one of the places where I see the conversation on social media among teaching and learning folks as being a little disconnected from reality, if I can say that, [LAUGHTER] because I think that there’s so much of this still going on. I think that we talk to each other and then we talk to these wonderful faculty who come out to our professional development events. And that’s just like the tip of an iceberg. And we don’t tend to talk to people who don’t come to our events, [LAUGHTER] or don’t engage with us on social media. Where I see in this Facebook versus Twitter differential is I see my friends who are in much more research-focused careers, and all of their demands and rewards are all based on their research productivity. So you say we’ve moved away from that, but I think we’ve moved away from that some, and we have higher teaching demands even on research faculty, but I don’t think that the focus on research for a lot of those folks has changed very much. And when you’re burnt out, and all of the values and reward systems are rewarding you for research and not teaching, and you have TAs doing a lot of the on the ground face-to-face stuff with the students. I think there’s a lot of these so-called traditional practices still going on. And so I think it’s happening, and I think using the word harm might be a whole other big conversation. I would like us to step away from that word when talking about individual faculty and the teaching choices that they’re making in their classes. I think that there’s a host of problems with that word. And I think that part of the backlash, too, is this feeling of feeling judged by teaching and learning center, folks, and I think that that is not going to get us toward the sort of change that we want if there’s kind of these scores of faculty who are not part of the conversation yet. They’re not going to step into the conversation, if they’re feeling like we feel that their teaching practices, that they feel like they’re doing the best they can, are poor or actively doing harm to their students.

Rebecca: And the reality is they’re probably not actually harming students. Students are still learning and they’re having some other experiences and they’re facing challenge. That’s not, as you noted in your article, that’s not harm. Can you talk a little bit about fostering challenge for students?

Sarah: I don’t like to equate high-stakes exams and all these things… this is challenge and then student-centered teaching is low challenge, because I don’t think… and I’m not saying that you’re doing that… but I just want to be clear on my position for that. I think having really high expectations of students is challenge. I think asking students to take emotional risks and be a little vulnerable is challenge. And I think that all of that is done most effectively in a classroom where community has been built where students feel that they belong. I think reducing stakes is a really powerful tool to developing that feeling of safety in terms of willingness to take risks. So I am pro low-stakes. [LAUGHTER] And I’m not arguing that we should all be going back to sage on the stage and three exams, certainly not. And I have worked really hard over the last 10 years to try to nudge us away from those practices. But, I do think that they are not necessarily causing harm. And I think most of us, in our intellectual journeys, thrived under those conditions. I do think that people who have thrived under those conditions, usually people who’ve had a lot of privilege and a lot of resources. And so that’s another reason why we need to move away from these practices. But I think if we want to pull people with us and reduce high-stakes exams and reduce some of these practices that may not have been as effective at bringing all students to the table, then we need to do so with an open hand and also have ways that faculty can make incremental change. I reference in the article, Michael Palmer had a POD presentation at one of the teaching and learning conferences, in which he presented 15.9 million choices [LAUGHTER] that a faculty member makes in the construction of an individual class and argued that if we really want to make lasting change, we should be nudging little changes in all of those choices towards student centeredness, toward more inclusivity, rather than asking faculty to kind of throw everything out and start fresh and do something entirely new, and I found that compelling. And so if we could do that nudging, in the absence of judging, and by suggesting that there might be better effectiveness and equity, and doing that nudging, rather than saying you’re actively harming students with your current practices, I think would be a lot more compelling for a lot of those faculty who aren’t necessarily coming to our workshops.

John: So, nudging not judging, that’s a nice way of summarizing that, I think.

Rebecca: Yeah.

Sarah: That is a much more catchy way of saying it. [LAUGHTER]

John: I was thinking it would be a good title…

Rebecca: Yeah.

John: …or the podcast.

Rebecca: I was just thinking about, you never know what small little change you might make, that actually makes a humongous difference for a learner.

John: To use the technical term

Rebecca: I only use [LAUGHTER] technical terms. [LAUGHTER] So I think what you’re highlighting is really important, because these small little changes can make a big difference for an individual learner, or a set of individual learners. And you just don’t know what little tweak you’ve made, and what kind of impact it might have made. I’ve certainly had the opportunity to have a conversation with a student where they tell me about something that made a really big impact for them. And to me, it was like something that was really small, like, “Oh, alright, well, I should keep that.” [LAUGHTER] But it’s helpful. And they all add up over time.

John: What can we do to bring some of these other faculty who haven’t been as active in discussions of teaching into conversations, and to make it easier for us to do some of that nudging? We can’t really nudge people if they’re not talking to us or not coming into teaching centers?

Sarah: Well, I think more dedication of resources from the administration would be good. I think that bringing people to these events who haven’t traditionally come might happen if there’s more time and money, a course release or something, aa stipend for something else. I saw a presentation by Betsy Barre, a wonderful presentation where they had participation in a one-day teaching celebration event that was at something like 67% of the faculty attended. But it was very well resourced. They had a bunch of really fantastic speakers, a lot of great food, it was an entire day, and the provost canceled classes and like kind of shut down campus for the entire day and said, “This is important, come to this.” And they got something like 67% of their faculty at Wake Forest to attend that. So I think more resources is an answer. I think also having more varied offerings. So I tell the anecdote in the article when I was at Assumption University, one of our faculty came to us and he was from a department that never attended anything [LAUGHTER] that we offered and he said that, and he said, “Actually, none of us ever come because we think that you’re doing harm to teaching with all of these approaches that he felt were not effective, and that were kind of degrading education.” [LAUGHTER] And we had a conversation, and so what the result of that conversation was we had one of our faculty learning communities that we do over dinner with a book, and he gave a presentation called “the art of the lecture,” and it was a defense of lecturing, primarily lecturing as a mode of pedagogy. And then we also had a few other faculty do little flash talks about how they did kind of dynamic innovative lecturing. And then we had a conversation and a debate. So we weren’t advocating for this, but we were showcasing him, and then faculty were engaging in debate and dialogue around when is lecture appropriate? Like what are the ways to make it more student centered. And I would like to say that his whole department came out to that, and they didn’t, [LAUGHTER] but he was there. And I think demonstrating that kind of respect for all the variations of teaching, I think is good. Another that we did at my current institution is we had a panel of faculty and we intentionally chose faculty with varied deadline policies, so on the spectrum of more strict, more flexible, and kind of in the middle, and had them present their rationale and their procedures and then did q&a. And that was really well received and better attended than some of our other events. So I think mixing things up and creating a space where it is clear that we are respecting faculty autonomy, and that we see variation as good. I think it can be powerful.

Rebecca: I’m hearing some themes of celebrating teaching and learning [LAUGHTER] and respecting and listening.

Sarah: Yes.

Rebecca: …listening and respecting a wide range of teaching in all of the examples that you just shared, which I think are two things to remember and keep in mind, because I think you also said not in this last answer, but a couple answers back something about [LAUGHTER] I think you put the words faculty and equity in the same sentence rather than we usually think about students and equity. But I was hoping maybe you might elaborate a little bit more on this idea of some equity around faculty as well when we’re thinking about teaching in particular.

Sarah: I think that probably what you’re referencing in my previous answers was when I was talking about Chavella Pittman’s visit. And I think one thing that we don’t always acknowledge is the impact that our teaching choices can have on other faculty in our department and at our institutions’ teaching choices. And so she was talking about how many more incivilities and classroom disruptions faculty of color and other minoritized faculty get compared to white cis het faculty. And that when we make choices in our own classrooms, that might have an impact on other faculty. And so specifically, when white majority faculty have a lot of policies in their classes, that they have a lot of flexibility, or they don’t have a classroom disruption policy, or the classic example is “Call me Sarah, and don’t call me Dr. Cavanagh,” that sort of thing. That actually makes it more difficult for faculty who, in order to have a good classroom climate, needs to demonstrate their authority. I know we don’t like to use that word when we talk about teaching, but all our friends need to be called by “doctor” and need to have a classroom disruption policy, because otherwise there’ll be a lot of classroom disruptions. If there’s also this perceived differential in those two classes, then those faculty are going to get accusations from students that :”Oh, you’re so strict, you’re so this or that compared to these other faculty who are just like, relaxed, why can’t you be more like them?” And so I think that we do need to think… I’m saying “respect faculty autonomy,” but then also, “let’s talk to each other about our classroom policies in order to make sure that we’re not engaging in these behaviors that are going to impact other faculty.” But again, it’s a dance, it’s a dance and a balance, a little bit of this, and a little bit of that.

John: And I’ll mention that Chavella Pittman and Tom Tobin had a discussion of this in an article in The Chronicle, I believe, and then they joined us on a podcast for that too. And we’ll share links to both of those in the show notes.

Sarah: This has been a great conversation, but I want to be clear that I am so dedicated to student-centered teaching. And that is why I feel really passionate about this backlash because Rebecca, you mentioned listening, and I’ve just been listening and listening. And I’ve been hearing it all over the place. And again, I worry with these pendulums that we could just snap back to a place that we really don’t want to be. But I think sometimes when I have these conversations, I come out as “Oh, pro high-stakes exam and let’s have three exams, multiple choice, strict attendance, policies, all these things.” And that’s not the case, I just think that we need to do the nudging, not judging so that we can continue pursuing change, because I think we had this accelerant with the pandemic, and it was great, with so many people in zoom rooms and in seats, and really eager to like innovate and all of these things, and that has kind of faded, and we now have these other tensions that are pushing in the opposite direction. And so let’s have some new strategies so that we can continue pursuing educational reform

John: And the strategies of having different people with different perspectives discussing it, is probably a really good way of getting people to state their opinions, but also to hear some things that might nudge them in another direction. So someone who only lectures hearing a little bit about the possibility of doing an interactive lecture would be a relatively small adjustment that they wouldn’t have heard if they hadn’t attended an event of that sort.

Sarah: Absolutely.

John: I think that sort of approach is a really good one that we should try doing more of here.

Rebecca: I think also having varied classroom experiences is healthy for students, it’s a good experience for students and helping students to see the value and having just different opportunities, different ways of flexibility being demonstrated to them is helpful to point out to students rather than being really careful when you’re having a conversation with a student about the classes that they’re in, for example, and highlighting maybe some of the skill sets they might be getting from a particular experience rather than kind of bashing a faculty member or particular class and helping them understand they might want a variety of different kinds of classes to help their own workload and interest and that kind of thing. So we always wrap up by asking: what’s next?

Sarah: Well, I have an NSF-funded network in intro bio called TUnE-Bio, and we’ve been holding a lot of qualitative interviews of intro bio faculty and and digging into some of this, they’re thinking about change and the constraints and barriers and facilitators of change and I’m getting excited about kind of expanding that network into the possible future.

Rebecca: That sounds like some interesting work.

Sarah: Thanks.

John: Well, it’s always great talking to you. Thanks for joining us. And we look forward to hearing more about that project as more information comes in, as well as anything else that you’re doing that you’d be willing to talk to us about.

Sarah: Great. Thanks for having me.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

349. Growth Mindset Messaging

First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, Elizabeth Canning, Makita White, and William B. Davis join us to discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU.

Show Notes

Transcript

John: First-generation college students, on average, have lower GPAs and higher dropout rates than continuing-generation students. In this episode, we discuss a growth-mindset intervention that has eliminated this equity gap in a large STEM class.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Elizabeth Canning, Makita White, and William B. Davis. Elizabeth is an Associate Professor in the Psychology Department at Washington State University. Makita is a graduate student at WSU’s Experimental Psychology Program, and William is a Professor of Molecular Biology and the Interim Vice Provost for Academic Excellence and Student Achievement at WSU. Welcome back, Elizabeth and Makita, and welcome, William.

Bill: Thank you. It’s a pleasure to be here.

John: Today’s teas are:… Elizabeth, are you drinking tea?

Elizabeth: I am drinking tea. I’ve learned from past podcasts with you that I should bring some tea along. So I’m drinking some sweet and spicy black tea today.

Rebecca: Sweet and Spicy… [LAUGHTER] sounds interesting.

Elizabeth: It has some cinnamon and some orange. It’s very good.

John: Very nice. And Makita?

Makita: I’ve got classic Earl Grey today.

Rebecca: Can’t go wrong with a classic. [LAUGHTER]

John: And Bill?

Bill:I have a Raspberry Zinger herbal tea today.

Rebecca: I’ve moved on to Awake tea… this time of day. [LAUGHTER]

John: We recorded another podcast much, much earlier today. So that’s probably needed a bit more. [LAUGHTER] And so that I won’t be awake all night tonight, as I have been for much of the last couple of nights, I have just a pure peppermint tea.

Rebecca: Probably a better choice for this time of day. [LAUGHTER]

John: And for this time of the year.

Rebecca: So we’ve invited you here today to discuss your March 2024 study: “Growth Mindset Messages from Instructors Improve Academic Performance Among First-Generation College Students,” which is quite a mouthful. [LAUGHTER] But before we discuss the study, can you talk a little bit about the difference in outcomes between first-generation students and continuing-generation college students?

Makita: Sure. So when we say first generation, we usually mean someone who’s the first in their family to attend college. For this study, we specifically defined it as a student for whom neither parent has earned a four-year college degree. And these students typically have to face and overcome a lot of social and cultural barriers in order to be successful in college. And although almost a third of the people who go to college are first-generation, they don’t always seem to do as well as continuing-generation college students. On average, they tend to have a little more difficulty adapting to college, they typically earn lower grades and they have higher dropout rates. They also act a little bit differently, they’re less likely to do things like go to office hours, or communicate with their instructors, whether that’s by email or in person to seek clarification on course materials. And just in general, they tend to be less likely to try to access helpful academic resources when we compare them with continuing-generation college students, which would be students for whom at least one parent has earned a four-year college degree. And on top of all of that, the students are also usually… I mean, they have to be… dealing with having less familial guidance than their peers do when it comes to navigating higher education. So if you’re a continuing-generation student, you have a parent who might tell you something like, “Hey, it’s a good idea to go to office hours,” or “You should be asking questions” or “there’s a financial aid department,” important stuff like that. They also may be dealing with a mismatch between the values and culture that they grew up in, compared with the values and expectations of an American university, which is typically very individualistic and may not be as supportive as what they are used to. And of course, they’re also more likely to be working, living off of campus, and they usually get less financial support from their families. But there are other factors that may exacerbate the difficulties that they’re dealing with. And this study is about one of those.

Bill:And one of the things that, just for context at our local institution to keep in mind, is WSU has a lot of, for lack of a better term, brand loyalty. It was not uncommon for me in my class to have fourth generation Cougars. And if you think about that legacy of your great-grandparents had attended this institution, as had other people, that sort of familial knowledge that gets passed along, is a huge benefit for that population of students as compared to the first-generation students as Makita was talking about.

John: One of the things that you mentioned, Makita, was differences in help-seeking behavior. And one of the studies on that is one that we talked about with you and Elizabeth in an earlier podcast, and we’ll include a link to that in the show notes. Could you describe this current study?

Elizabeth: Yeah, so this study we conducted in 2021, I believe. It was the second semester online, so the pandemic year. I know we’re all sort of crossing out that memory from our minds… [LAUGHTER] …that whole year. As instructors, it was a little tumultuous. But Bill and I got together and we designed this study together in his class. He was teaching at the time the introductory biology class at WSU. And what we decided to do is test out some actionable techniques that instructors can do to communicate a growth mindset to their students. So a lot of the research that my lab does looks at how growth mindset messages impact different students from different backgrounds. And at this time, there was a lot of suggestions about how to implement growth mindset ideas in your class, how to change course material, how to integrate it throughout, but there wasn’t really any experimental evidence of specific strategies that instructors can do. So we decided that we would just test it out in Bill’s class. So what we did in this study is we randomly assigned everybody in the course to one of two conditions: a control condition or a growth-mindset message condition. And after the first two major exams in the course, students received an email from Bill, and in the control condition they received a message that is pretty typical of an instructor after an exam has been conducted in their class. It says things like: “exam grades are posted online, here’s how we calculated them,”
“here’s what it means,” and then “feel free to talk to me in office hours, or ask questions about it.“ In the growth mindset message condition, we included all of that information, but we also included a fairly lengthy paragraph… I’ve gotten comments on… but it’s a big chunk of a paragraph that basically says, in Bill’s words, “Here’s what I believe about student performance and exam performance and improving in the class.” And there we integrated growth mindset theory, we talked about how abilities can be improved, we talked about how academic struggles are normal to experience, and that this struggle is part of the process of learning, that is something that is controllable. That you can put in effort that you can use different strategies to make improvements. And I think we used, Bill, if correct me if I’m wrong, we started with a draft of this email that you had already been doing in your class. And we just sort of infused it a little bit more. So Bill really wrote like the first draft of it, and then we kind of went back and forth a little bit to edit it and infuse it a little bit more with growth mindset. But it’s really written in the voice of the instructor. After we implemented this email after the first exam, we did it again after the second exam, changing the language a little bit. We assessed their performance across the semester. So we looked at their exam grades, we looked at their final course grade, and we found that students who were in the growth mindset condition earned higher grades in the course. And this was predominantly driven by first-generation college students. So first-generation college students performed much better in the course, when they received those growth-mindset messages from the instructor.

Rebecca: Can you talk a little bit about the effects and how significant they were?

Elizabeth: Yeah, so when we look at the group differences in the control condition, what we find that’s very consistent across big STEM courses across the university, that continuing-generation students are getting better grades than first-generation students. So in this course, they’re earning about three-fourths of a letter grade higher than first-generation students. So this is a really, really big gap between the continuing-generation students and the first-generation students. It’s not specific to this one course. So we see this in a lot of different datasets. But what was really remarkable in this study is that that difference in their grades was completely eliminated in the growth-mindset condition. So first-generation college students earned about three-fourths of a letter grade higher in the growth-mindset condition compared to the control condition. So this is a pretty large effect on final course grades. And of course, it’s pretty significant in terms of the curriculum that they’re a part of. Bill, if you want to talk a little bit about the students who took this course and how it plays a role in their career trajectories.

Bill:Yeah, thank you for that. I was looking forward to jumping in a little bit. And this is a little bit about my journey too, in this course. So, at this point, just for context, this was the 16th semester that I had taught this course. Largely I had been either the sole instructor or I was the primary instructor of record for the course. It’s taken by about 450 to 550 students each semester, little bit of differential between fall and spring in size, over 50 different majors, or pre-major students take this class. It’s nominally a freshman majors biology course. But there’s a substantial number of students who are neither biology majors or life science majors. And there’s a sizable number of students who are not freshmen. So, in reality, the highest population of my course normally were sophomores, freshmen were the largest population in that spring semester. And that was pretty typical, because it is oftentimes taken as the second semester of an introductory biology sequence. So yeah, and to be honest, over the years before this study, I had been trying to narrow those opportunity gaps between first-generation and continuing-generation students. I knew they existed. I had done many things with the course experimentally over the years, and never had really closed that gap. This was one of the first times, if ever, I saw that gap completely close in anything that I tried, and I tried tinkering in the lecture in the lab, big changes, small changes, a lot had gone on before this.

John: You mentioned that the class is normally about 450 students. Could you tell us a little bit more about the specific course in which you implemented this? You noted that this was during the pandemic. Was this offered remotely? Was it synchronous? Was it asynchronous? And how many students were there in this particular sample?

Bill:So, the course was taught in a synchronous Zoom manner. So both the lecture and the lab were taught using zoom. I believe, if memory serves, I had the largest Zoom Room on campus that semester. I can’t remember the exact number, I want to say my enrollment that semester was somewhere close to 550, or 600

Makita: The final sample is 417, I believe.
Bill:417. That’s right. That’s how many students, so it must have been around probably 450, at the beginning of the semester, or something like that. Yeah, we had synchronous Zoom every day with students. In fact, I learned pretty quickly how to crash Zoom my first semester I taught remotely. [LAUGHTER] So by then, it was a little bit more of a pro, and at least knew how to keep it running for 50 minutes at a time.

Rebecca: And true badge of honor.

Bill:[LAUGHTER] Well, there are many lumps that we took in the pandemic and learning how to teach and Zoom was one of them. [LAUGHTER]

John: I had 350 students in the fall of 2020, and it was not my most enjoyable teaching experience. What are the factors you controlled for? Because you did control for some other variables to separate out the effect of the growth-mindset messaging?

Elizabeth: Yeah, so we controlled for three things in our analytic models. We controlled for students’ prior college GPA… so this was just their self report… GPA that they’ve had in college. Since this was in the spring semester, all the freshmen had had a semester under their belts at that point, to have an idea of what their college GPA was. We included this just because we were looking at performance. So we wanted to filter out or control for their prior performance, that’s a big predictor of their performance in the class. We also controlled for their self-reported personal mindset beliefs. So we asked the students themselves, what do you believe about the nature of intelligence? Do you think it’s something that can improve? Do you think it’s something that you’re just born with? So we measure their own self-reported mindset beliefs, and we control for that and all of the analyses, because we want to look at the effect of the intervention above and beyond student’s own beliefs about ability. And then the last thing we controlled for was their race or ethnic status. So what they self reported to us on a survey. Here, we controlled for that because there is some overlap between race, ethnicity, and first-generation status. We did look at the different groups separately, we don’t have enough power to really look at race ethnicity as a moderator at the intervention. So we control for it in our analyses to look at the effect of first-gen status above and beyond race and ethnicity,

Rebecca: I can imagine, as an instructor who’s fiddled quite a bit in their class to overcome these equity gaps, that it might have been a little surprising [LAUGHTER] how much of a change happened with such a relatively small intervention.

Bill:It was shocking. It’s funny, because one of the things that Elizabeth has really talked about is how we took materials that I actually already possessed in some ways, and modified them slightly in some cases, and more radically in others, but just that transformation was enough to start to really make an impact. And you could see it happening. And that was amazing to me. I remember when I halved my opportunity gaps in my class with an intervention. And I remember dancing and celebrating, [LAUGHTER] and then this one was just on that scale. This was huge and amazing to me.

John: One of the interesting things in this study is you were looking at the effect in terms of grades on subsequent exams, but you didn’t find much of an effect on the second exam from the first intervention. But when you had two of those messages coming out, that’s where you found the relatively large effect. Why do you think the second message was having more of an effect than just a single message?

Elizabeth: Yeah, so we have a couple of theories. We think that one reason could be that they just need the message more than one time. So they need two doses or more than two doses to sort of really get people’s attention and make them change their behavior. That c ould be one reason. Another reason is that it might be enough to just have one message, but it would take time for behavior to change in order to see a difference in their performance. So it could be that after the first exam, they started engaging with the material differently and then that then sort of snowballed to affect their third exam scores, and then again, their final course performance, but we don’t really know exactly what’s going on. Another reason could be that towards the end of the semester, students get a little antsy, and they start waking up a little bit to, “Oh, I should probably be doing something different if I want to have a grade that looks like this.” So it could be timing, it could be dosage, it could be the effects take longer to materialize in terms of performance changes. One of the interesting things that we found here in this dataset is we did try to look at behavior in the way that we had access to, so we pulled all of their activity on the course website. And this semester, our campus was using Blackboard as the course sites. And we pulled all of the Blackboard data. So we looked at whether students in the growth mindset condition were engaging with Blackboard differently. And we found that they were. Students, on average, who were in the growth-mindset condition engaged with the course material 40 instances more, on average, than students in the control group. This is around a 12% increase in webpage engagement. So after they get this message from the instructor saying that intelligence is something that can grow, abilities can improve over time, and here are some strategies that you might implement to realize those changes, they’re going to the course website, they’re clicking on things, they’re re-reading their notes, they’re looking over the course lectures, they’re engaging more with the resources that are posted there. And then not surprising to any instructor ever, [LAUGHTER] that when students are engaging with the course materials that then lead to greater performance in the class. So we found a mediation effect through their behavior with the course material that was posted online. So this is one example of the behavior change that I was talking about earlier, that when they get this message, it can be inspiring, it can inspire a different type of behavior that can lead to better performance in the course.

Rebecca: Those are really strong click rates generally, [LAUGHTER] especially when we’re often complaining that students don’t read our emails or messages. [LAUGHTER]

Elizabeth: Yes. So, we did look at differences in Blackboard in terms of clicking on course material versus going to the page where all their grades were, like the gradebook. And we didn’t find any differences on their engagement with the gradebook. So it wasn’t that they were just tracking it more or anything like that. But it seems to be the case that they’re actually engaging with the substantive course materials more, on average, in the growth- mindset condition.

Rebecca: It’s just incredible. [LAUGHTER] Can you talk a little bit about… and we’re hinting towards this, obviously… what does this study suggest that faculty and institutions can do to improve outcomes of first-gen students, or maybe even, Bill, what you’ve continued or started to do?

Bill:I think, for me, what I took away from it was being very intentional with my students in communicating what I believe about them. And I mean that in the spirit of the old NPR show, or the segment that they used to do, you know, “This I believe,” where people would have those small snippets. I felt like this intervention was something along those lines where I could be authentic in communicating what I truly believed about their abilities. And I did it in a way that, with some small tweaks, obviously had these much deeper and larger payoffs. And I think that’s one of the real nuggets of wisdom here is that when you want to transform, and have better student outcomes, you want to narrow these opportunity gaps for students from certain backgrounds or experiences when they come in. Oftentimes, you don’t have to throw away what you’re already doing. Oftentimes, you already have nuggets of really important things that you’ve built over time. And all you need to do is to work with somebody who sees things just a little differently and can tweak them with you. And I think that partnership with Elizabeth and Makita was really critical for me. They were able to take things that I already had, and show me a different way to just modify what I was already doing. That messaging would go out to every student after that time point. When you see an effect that size, it almost becomes ethical to in fact, it is ethical, in my opinion that you don’t preclude any student from experiencing it. So that should be the way that everyone should be communicating with their students, in my opinion. And you want to make sure that it’s authentic as well, because I started with a set of materials that I had already written and sort of came from my heart, my own experience, that made the authenticity, I think, real whereas if I would have tried to have said something to students that I really truly didn’t believe about their abilities, I think they would have sniffed that out in a heartbeat. And we probably wouldn’t have seen the effect that we did.

Rebecca: It’s amazing what a little bit of intention can do in the design of something, especially because we often don’t realize how big of an impact a simple communication might have. We almost think of it as like a throw away, or a quick thing you might be doing, but you add a little intention to it, and it starts having a really different impact.

Elizabeth: Yeah, I think you mentioned earlier that students might not read the email, you might think that they’re not going to engage with it. And we also thought that, [LAUGHTER] so we think that in designing this intervention, we were really drawing on a lot of the intervention research that had been done previously. So research on wise interventions, mindset theory, all of that was integrated into the design of the message itself. So we started with Bill’s original emails that he was sending out to students and the messages that he had already curated for his class, and timing it in a time in the class in which students are receptive to that information, I think, is also really key. So we don’t know this empirically. But I would guess that if we were to send that email a couple of weeks before the exam, it would not have the same kind of impact. We send it directly when that performance information was posted online. And that’s when students are questioning themselves. They’re saying, “What does this score mean? Does this mean that I’m not good at this? Does this mean that I can continue? Does this mean that I’m smart or not smart? …and that uncertainty, that is the right time to infuse your message as an instructor to counteract those uncertain messages about ability. So I think the timing really matters. I think authenticity, like Bill said, really matters. Makita and I have been working with other institutions to implement a similar type of intervention with instructors. And before we did that Makita did a bunch of focus groups with the students at these institutions and showed them the messages that we used in Bill’s class, and we ended up changing them quite dramatically. The students gave us feedback like “nobody here would ever say something like that,” or “here’s what you need to say to reach students here in my class,” or “here’s what I will be receptive to.” So students in different contexts are going to respond to different messages. And so you really do want it to fit your own class, how you would speak, how you would talk to your students, the needs of the students in that moment might be different from the class they’re in. So it really is not a copy and paste type of situation where you were gonna copy what we did here and put it in your class. I think you do need to kind of go through the process of generating it for yourself and your own students.

Rebecca: I could imagine there would be an element of needing to evolve it over time, too, as your student population changes, and as you change as a teacher.

Elizabeth: Yeah, absolutely.

John: One of the things that I think is particularly nice about this study is that we see higher DFW rates by first-gen students in STEM classes in particular, and so many students end up leaving STEM fields that they intended to major in in their first couple of years. And that’s where the rate of return to education is the highest. And in general, first-gen students come from lower socioeconomic groups and lower family income groups, and if we do want to see more equity in outcomes, getting more first-gen students into the STEM fields could do a lot to help reduce some of those equity gaps in terms of overall outcomes.

Elizabeth: You mentioned the long term nature of these effects. I think it’s a great question. I think it’s something that we’re currently looking at in a project that we’re doing that’s funded by the National Science Foundation, where we’re looking at how these interventions in an introductory course can change career trajectories, that when you get a B instead of a C in a major introductory course, that’s gonna mean something different to that student, that’s going to set them on a path of following that next course sequence, that next step. So that’s the question that we’re asking and the research that we’re doing now, and hopefully, we’ll have some answers in a few years.

John: Peter Arcidiacono at Duke had done a study on the decision to stay in STEM fields. And he did look at some of these questions in terms of the effects on major choice based on the grades that were received. We can share a link to that study in the show notes. In the IRB proposal, did you include the possibility of following up these students in terms of their outcomes a few years later, in terms of retention rates for the students?

Elizabeth: That’s a great question. I believe in terms of IRB, we had students consent to their academic records. So we should be able to go back and look at transcripts from the students who participated in the class. We could look at course taking, we could look at graduation rates, we could look at major. If these students were freshmen/sophomores in 2021, then we’re probably around the time where they would be finishing up so maybe in the next year we could.

Bill:I was just thinking the same. Now’s the perfect time to go back and take a look, because we’ve had enough time since the intervention to see what’s happened. And actually I was gonna go back to one of your earlier points. You were talking about statistics earlier. Basically, this is really the only way that we’re going to increase the number of STEM students going forward. If first-generation students at WSU are 40% of our population today, that means if we don’t improve their outcomes, then we’re going to continue to see the declines in STEM outcomes and STEM graduates that we are trying so desperately to fight against, and also, we’re not going to have the representation in our STEM graduates of the population of the US that we want to see.

Rebecca: I think in addition to the growth-mindset piece, I’m sure that the authentic nature of the messages also just showed deep care. And that can be really important for first-generation students who need that ally or advocate for them in their college journey. So I can imagine those messages coming from Bill in Bill’s voice that make them feel like “Yeah, you can do this, I care about you,” from that perspective is also really important.

John: And there was a study done by a couple of economists, entitled “My Professor Cares,” where they did follow up students four years later, and they did find a significant effect in retention rates four years later by a similar type of intervention, I don’t think it was quite as much growth-mindset focused, but just sending a signal that the professor cared about the students outcomes, a very simple message, made a significant difference in student’s grades as well as longer term outcomes. So it’s nice to see more studies of this, especially focused on issues such as growth mindset.

Elizabeth: Yeah.

Rebecca: So we always wrap up by asking: What’s next?.

Elizabeth: So Makita and I and our lab have been working for the past couple years on following up on these data in different institutions. We used this study as pilot data in a grant application. And now we’re sort of scaling up to other institutions. We have a project with a really large HBCU, a Hispanic-Serving Institution, WSU is continuing to be involved as well, an Asian and Pacific Islander Serving Institution to where we have a number of instructors who vary in different characteristics, different class compositions, and we’re looking at these instructor messages in an experiment in these different contexts to see what matters. Does it help students develop their own growth mindset? Does it change their behavior? Does it work in some contexts, but not others? Do the instructor characteristics matter? So far, in the past two years, we’ve collected data from around 10,000 students who have been involved in this intervention project, and we’re currently in data analysis phase. So stay tuned for future studies and papers that come out of that data set.

John: Makita?

Makita: Well, what’s next for me is Elizabeth and I are actually going to meet right after this to do a debrief and to discuss some of the analyses that I just ran on a study that will be involved in my dissertation.

Rebecca: And how about you, Bill?

Bill:For me, I think about: “How do I proselytize about this work?” Because I think about what little changes make a big difference in an institution. We talk a lot about student retention, we talk a lot about student outcomes, we talk about creating the global citizenry that we want to see emerge from higher education. And the one experience that every student has in common, whether they are a first-generation a continuing-gen, or from some other background, is they have to take classes, and they have to be in classroom environments every day of the week, for the most part, over a course of four years. And I think about what would it look like to have interventions like this, that have evidence that back them, that students get inoculated with repeatedly over multiple time points with different voices, and that faculty engagement in the process of helping our students become better, to find success and to move forward in their careers? To me, that’s the magic of this. I’ve been talking to other people about it and I think about what would a curriculum look like that had small things like this? And how could it transform what we do? And I think that’s a vision that I can buy into, and I hope others are gonna come along for that ride.

Rebecca: I think the scope of the interventions that you’re talking about, that have this kind of impact, are really easy to buy into. Because usually you’re thinking about some drastic, giant investment of time, is what most people are thinking about. And that often is what prevents action from happening. So I’m excited, excited to share this and to hear what comes of your bigger study.

John: And one thing we appreciate is when we see things like this posted on social media so we can hear about it [LAUGHTER] and we can share it out a little bit further. [LAUGHTER] And I hope you’re presenting this at various conferences to reach a larger audience.

Elizabeth: Yeah, we will be presenting some of this work at SABER this summer. So that is the Society for the Advancement of Biology education research, and we’ve presented it in some other psychology conferences throughout the past year as well.

John: Excellent.

Bill:And I gave a plenary talk at the summer educational meeting for the ASBMB, American Society for Biochemistry and Molecular Biology. So I gave a little preview of what we had found. And that generated a lot of discussion and excitement, actually people asking for additional resources and ideas and things. So I think there’s a receptive audience out there.

Rebecca: That’s wonderful.

John: Well, thank you for joining us. It’s always great talking to you and it’s great to hear about this newest study and to meet you, Bill. We’re looking forward to hearing more about how things go in the future.

Elizabeth: Thank you so much. Thanks for having us.

Rebecca: I hope you’ll reach out when your next study is done. [LAUGHTER]

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

348. Active Learning Initiative at UGA

While there is compelling evidence that active learning results in increased student learning, these initiatives often face resistance from students and faculty. In this episode, Megan Mittelstadt and Leah Carmichael join us to discuss the active learning initiative at the University of Georgia that provides professional development for faculty, active learning training for students, and for the redesign of classroom spaces. Meg is the Director of the Center for Teaching and Learning at the University of Georgia. Leah is the Director of Active Learning, also at the University of Georgia.

Show Notes

Transcript

xJohn: While there is compelling evidence that active learning results in increased student learning, these initiatives often face resistance from students and faculty. In this episode, we explore a large-scale active learning initiative that provides professional development for faculty, active learning training for students, and for the redesign of classroom spaces to better support this initiative.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Megan Mittelstadt and Leah Carmichael. Meg: is the Director of the Center for Teaching and Learning at the University of Georgia. Leah is the Director of Active Learning, also at the University of Georgia. Welcome Meg: and Leah.

Meg: Hi.

Leah: Good to be here.

Rebecca: Today’s teas are:… Meg, are you drinking tea?

Meg: I am, I’m drinking double chai. Tea is a big deal. In our center. All of the people who work here have favorites. And we provide tea service at every workshop and I checked this morning, the most popular tea at our workshops is lemon ginger.

Rebecca: Oh, that’s good to know. Another teaching center full of tea. [LAUGHTER]

Meg: Yeah, we even have the teas of folks who don’t work here anymore that we keep stashed, just in case. So, Leah, you know, Colleen. Colleen drank…

Leah: Yeah.

Meg: …caffeine free peppermint tea, and we still keep it stashed, even though she doesn’t work here anymore. [LAUGHTER] Just in case.

Rebecca: How about you, Leah?

Leah: Well, I am a diehard coffee drinker. And so I am enjoying a cup of coffee. But my best friend is a potter, so I have her beautiful mug here to enjoy it with.

Rebecca: Wonderful. How about you, John?

John: I’m drinking a black raspberry green tea today.

Rebecca: Oh, that’s nice. And I’m back to my blue sapphire black tea.

John: Black and blue?

Rebecca: It’s black and blue. [LAUGHTER] We’re recording this at the end of the semester. [LAUGHTER]

John: We read about the active learning initiative in a recent Chronicle article and we invited you here to talk a little bit about the active learning initiative at the University of Georgia. We were really impressed to see that the university had budgeted over a million dollars a year for five years as part of the university’s Quality Enhancement Plan. This is a really large investment in effective teaching practices, and it’s especially impressive that this is happening at an R1 university. How did this initiative come about?

Meg: Yeah, we’re really excited that this investment is happening. And it kind of stems from an initial pilot that happened in 2018. There was a presidential initiative on active learning that resulted from a faculty task force. And so in 2018, our president allocated a million dollars to active learning programs, as a pilot. I feel super lucky to have a president who cares so deeply about the teaching enterprise at UGA. It’s something that you don’t see at every institution. And so I really appreciate it, value it, and count my luck there. Our president allocated a million dollars in 2018 for an active learning course redesign institute and a pilot classroom infrastructure renovation, and also the creation of a one-credit course for our students called Becoming an Active Learner that’s deployed by our Student Success Center. And that first year, we thought, “Gosh, it might just be a one-year investment, so we really got to make this count.” And so all the folks in the center sort of got together and backwards designed the programming for our first Active Learning Summer Institute, which by the way, was one of the most gratifying and enjoyable experiences of my career to get five educational developers in a room and get to go totally meta and apply the backwards-design process to the creation of a Course Design Institute. We nerded out. [LAUGHTER] We had like the room where we had all of our notes on the walls, and it was sort of our Beautiful Mind room for the Course Redesign Institute. We loved it. And so we designed this institute for 2018, and it went quite well. The folks at our institution who applied were way more applications than we could accommodate in the Institute and the folks who got into the Institute had a good experience. And we were able to show a couple of courses that had been redesigned seeing some really significant positive outcomes for students, both in terms of their depth of understanding, but also in some of the metrics in courses like numbers of Ds, Fs and withdrawals from courses decreasing in a few really key courses like our introductory mathematics course sequence. And so we were able to go back to the President and the Provost every year after that and say, “Okay, here’s how the course redesign Institute went, could we get funding to do it again?” And so we continued to do it every year. And this continued for a couple more years. And when we got into the 2020 timeframe, UGA was preparing for our next reaffirmation through SACSCOC. And for SACSCOC reaffirmation, an important component of that reaccreditation proposal is the creation and proposal of a Quality Enhancement Plan (or QEP). And so a committee was assembled to come up with what that topic might be, and they looked at programs that existed on campus as well as the university strategic plan. And luckily, by this point, our strategic plan had been drafted with quite a few key performance indicators related to the adoption of active learning practices. And so they identified this opportunity to scale the Active Learning Initiative at UGA. And so they proposed this topic, it was selected and now the funding has been stabilized for some of the pilot programs that existed, but also some newer initiatives to0.

Rebecca: There’s a substantial body of work supporting the efficacy of active learning and increasing student learning and reducing equity gaps. And it’s so exciting to hear about such a big investment and the interest of faculty [LAUGHTER] having more interest than spots for example, But I know one of the things that sometimes happens when faculty decide to start implementing active learning in their classes is sometimes there’s some pushback from students, but I heard you say something about a one-credit course for students. Can you talk a little bit about that? And how that ties to preparing students for this endeavor?

Meg: Yeah, so because we had this course from the get go, there was already this idea that reaching out directly to students and building their capacities and dispositions as learners to amplify their success in an active learning environment is something we might ought to think about. Of course, we also think about it from the faculty side too. One of our most popular sessions in the Course Redesign Institute is a session on student resistance to active learning, and we rely heavily on Tolman and Kremling’s Why Students Resist Learning for that session and then talk about some of the instructor facilitation and explanation strategies that instructors can use to ensure that student resistance is not cropping up and to help prevent barriers to student engagement. I think what’s really cool about Tolman and Kremling’s book is that they reframe “student resistance” to “barriers to engaging” and so we work with faculty to reframe student resistance as “Where are the barriers to engaging that we need to address as instructors?” And then on the student side too, the philosophy is “Where are there barriers to engaging?” And then “When can we do some really important work to help students appreciate that engaging in active learning is a really actually durable practice for them?” It’s really going to benefit them long term. And since that idea was already in the mix, as we were coming up with the programming for the QEP, there was a really large committee, I think, a 33-person committee of faculty, students, and administrators who proposed the programming that would go into our Active Learning Initiative. And one of the things that they wanted to think about was the role that metacognition plays in active learning. That was identified really early on because we were already aware of the student piece, thinking about metacognition as a key component of active learning. The committee spent quite a bit of time unpacking: “What are the learning dispositions involved in student metacognition? And what are the learning dispositions involved in lifelong learning and other benefits of active learning? And where’s the overlap?” So we looked at things like the AAC&U value rubrics on lifelong learning. And I think the other one was, I want to say, it was like transferable learning, I’m getting the title wrong. But anyway, there were two AAC&U value rubrics that we looked at, as well as some of the literature on metacognition and literature on lifelong learning. And our assessment team identified four student learning dispositions that seemed to exist across all of the categories that we were looking at: students acting more curiously, students engaging with initiative, students becoming more reflective in their practice as learners, and students drawing connections between what they were learning in their active learning courses and developing other knowledge and skills. And so we use this framework of these four learning dispositions to really drive our student programming. And I’m going to pass it over to Leah, so she can share a little bit more about how we’ve expanded beyond this one credit hour course to a whole portfolio of student facing initiatives.

Leah: So, first of all, I would like to say with the instructor development piece, I was an instructor on campus, a lecturer, and was in that first cohort in 2018, an international affairs scholar. And it really was the first time that I had had such a professional development opportunity that felt like it truly revolutionized the way I taught, the way I thought about my students, the way I interacted with my students. And so whereas I was taught to transform one of my courses, I ended up transforming them all. And when I was able to be on the team for active learning, I signed up, it was really excited to do that. So it was an amazing experience. For the student piece, what I really love is that we have not just gone to the instructors and said, “Alright, it’s time to consider something different and then your job is to not only do all this hard work to change and rethink your courses, but then can you do us a favor and sell it to the students?” That’s just such a burden and very difficult. And so within CTL, they do a great job of training us in the Institute to go in first day and frame the course, to craft our syllabus language in a way that would potentially be able to introduce this course before even day one and to help decrease that student resistance. So that was a really important piece. But we also would have these teams of students, these just superstars on campus, who are also part of the programming for active learning. One of my absolute favorites is the peer learning assistants. This is an on-campus paid position for students. They apply and they are able to either return to a class or return to an instructor whom they have worked with before. And they attend a class where they facilitate active learning approaches and activities with the instructor. So the peer learner is responsible for helping with the planning of the course and preparation with the instructor implementing it and then also of course, following along with the course materials. And this is a key: this near-peer experience has been key in allowing the burden to be shared in introducing active learning within a classroom. And the students look at these peers, maybe a semester ahead of them sometimes, maybe years, and they start to see these mentors that are built in. And these students know and care about active learning. They see why it works in the class and they’re returning to the class. And we have found that those classrooms have been just the most wonderful place to see this cultural change among students. We also have active learning ambassadors. And they are trained to do outreach. They do demonstrations for faculty. One of my favorite moments is we have an active learning summit every year open to the public. But we have an active learning summit in which they run a showcase where the faculty walk in, and there are students behind tables with interactive material demonstrating different active learning techniques and how they see it in their class, how they use it for study skills, and how they would enjoy instructors being able to incorporate it in their classroom. The faculty love it. They stop, they ask questions, they think, “Well, what about this? Well, this would be difficult.” And then students give that experience from their perspective of how these techniques work for them. And we see that the faculty really crave that and really enjoy being the learner in that moment, and learning from a student about what techniques work best for them.

Meg: And Leah, I think one thing that’s really unique about our quality enhancement plan that I haven’t seen at other institutions yet, or if they have them, I haven’t seen them, so mea culpa. But one thing that’s really cool about our Quality Enhancement Plan is that we have a partnership with our Office of Student Affairs where they’ve designed some programming through our residential life curriculum to embed active learning in the co-curriculum too, so that students are experiencing and talking about the value of active learning not just in the classroom with each individual instructor, but experiencing active learning outside of the classroom in the co-curriculum. Students who are leaders on campus are trained in active learning facilitation techniques, so that when they go in to serve as leaders of student groups on campus, they have some of these active learning tools in their back pocket that they can deploy and experience on the other side as well. So we’re really seeing active learning proliferated, not just in the classroom, but outside of the classroom too.

John: It’s nice to see such a holistic approach. In so many cases where we’ve seen active learning initiatives, it’s usually just designed for faculty development, to go and fix all those issues and challenges. And by approaching it from all these angles, it seems to be a really effective way of addressing some of the challenges that faculty face when doing this. One of the challenges, though, that faculty sometimes face when they introduce active learning is they go into a classroom where the seats are either bolted down, or are very rigid. I think part of your plan involves some type of infrastructure changes to support that. Could you talk a little bit about how classrooms have been redesigned as part of this process?

Leah: It’s such a key aspect of it, I have the anecdotal experience where I graduated from the Active Learning Summer Institute, went into a classroom ready to be highly engaged and interactive and set up have a highly mobile class design only to find that I was in stadium seating, with fixed seats and I thought, “Oh, no, what am I possibly going to do?” And so we have three foci, within the initiative. And we’ve mentioned the instructor development piece. So key, but as you said, it can’t stand alone without support for the other two. We have the student piece, we also have the classroom piece. And so our goal is to every… usually it has to be in the summer, when it’s much quieter… every summer, we just push for classroom enhancements across campus. And we have three criteria in mind that we see as not necessarily perfect for every single classroom situation, but the baseline for which we want to capture for an active learning classroom. And so the first is: does this classroom have mobile furniture? Are we able to move students around? Could an English professor have a roundtable discussion? And then within the 20-minute transition time, could my students go into that same classroom, which happened last semester, in a highly mobile classroom and be able to set it up into a simulation of roleplay thing for my international politics course? So do we have classrooms with that mobile furniture is the first thing. The second is, do we have student collaboration tools? And these do not have to be high tech. In fact, we ran the survey, and students and faculty said just give us more dry erase boards. We don’t need the fancy, fanciest of smart screens. Please just give us the basics. But let us work on a project huddled over together and be implementing and creating something new. And then finally, it’s just general space, do we have enough space that we are able to not only move the furniture around, but move people around in an effective way. And so 25 square feet per student is kind of our ultimate goal. And so we looked around and I was actually just running the numbers recently. And I think on campus we have, depending on how you measure our classrooms, between 400 to 600 spaces, it depends on if you count labs, and these other learning spaces, about 82% have at least two of these criteria. And so we’re getting there. We’ve made a real push and we’re trying to say: “Yes, instructors, please change but also know that we’re going to meet you with an environment that facilitates this kind of approach to learning.

Meg: Well, I also have to interject and say that the stance of the CTL is that active learning can happen in any classroom no matter the features. And so of course, like a good backwards design course or re-design institute, we always start out with the situational factors of a course. And that might include what’s the likelihood that you’re going to be in a fixed-seat auditorium or fixed table, but moveable chairs, and think about the choreography of that, the active learning techniques that you’ll adopt based on that. But in Leah’s example, in particular, if I remember correctly, Leah, you had a student in a wheelchair in your course the first semester after you redesigned your course. And when you were in the tiered auditorium, that meant that there was a specific area where the student in the wheelchair could sit and they couldn’t move through all of the case study roles that you had available. And so you were able to advocate for a different classroom, you obtained one, thank goodness, and everyone could move through a flat classroom instead. And so sometimes things crop up, even if you have planned to be in a fixed-seat auditorium, that you have to be able to modulate on the fly based on the characteristics of your students each semester too.

Rebecca: Can you talk a little bit about whether or not your program has focused on specific programs or if it started with specific programs and its spread to different programs or what your strategy is for involving faculty across disciplines?

Leah: One of the most important parts that I really liked about this initiative is it’s not a mandate, we are not requiring any instructor to adopt active learning through a mandate. Instead, it really is a support, educate, encourage. And so we really have not been focused on okay, this unit by X year needs to have Y outcome in terms of an active learning goal. But in terms of strategy, what’s nice is we can start to see the people that walk in the door to find the active learning Summer Institute. And the high demand that first year was a nice little kind of bellwether that, okay, there is something going on here. When you have more people that can fit, and that’s consistently. The CTL needs to keep it kind of a quality curated experience with a cohort that goes through and has to do these really difficult re-education, understanding education differently, and then of course redesign. Such a dense set of tasks. So with this cohort like experience really requires so much of the faculty. What we started to notice, as we looked around and realized that instructors from different schools and colleges were coming to the Summer Institute, and we have seen that as the CTL has expanded beyond the Summer Institute and provided workshops, both standalone workshops, as well as a course redesign experience outside of the institute. We see hundreds of faculty each semester coming into the programming of active learning. And really we have, I believe, 19 schools and colleges that have undergraduate programs, and we have representatives from each that participate in this programming.

Rebecca: That’s wonderful.

John: What type of training do faculty receive as part of the institute and other training that you provide?

Meg: It was initially just that summer Course Redesign Institute. And what we started hearing from faculty as we would get these applications every year is the deadline would pass, and we’d hear from some faculty who would say things like, “Oh, I really, really want to do the institute. but summer is my field research season” or “I’m in the clinics in summer,” or “I have childcare duties that I can’t get away during that time of year.” And so there started to be some sort of advocacy for other on ramps into the programming, so that when we had the opportunity to stabilize the funding through the Quality Enhancement Plan for active learning initiatives, we started thinking about how can we make this program one that faculty could find the on ramp that is appropriate for their needs? Is it a faculty member who’s been using active learning for a long, long time and they really want to brush up on their skills in one particular area? And so that might look like engaging in a specific workshop on something like facilitating group discussions in a large lecture environment, or looking at student metacognition and how that plays into a particular student developmental level that they might teach in their courses? Or does somebody want to do the whole workshop series and Course Redesign Institute all the way through, but they can’t do it during the summer, or anywhere in between. And so what we ended up with is a workshop series, we have 12 standard workshops that we offer over time: four fundamental workshops, and eight special topics, workshops, and we rotate through those. We offer a couple of each every year. And then we have the active learning summer institute that happens every May. And then starting next academic year, over the course of the spring semester, we’ve designed a course redesign experience that takes the course redesign elements out of the active learning Summer Institute and decouples those elements from the active learning workshop components of the Course Redesign Institute that we currently have, so that faculty can upskill through the workshop series. And once they’ve taken eight of the 12 workshops, they can apply for the course redesign experience over the spring semester. Through that course redesign experience, they come and meet with us for a couple of hours every three weeks or so over the course of the spring semester. And the idea is that by the end of that course redesign experience and workshop series completion, they’ve recapitulated all of the experiences that you might have during the multi-week summer Course Redesign Institute. We also provide different communities for folks to engage with. We have several robust Faculty Learning Communities on campus. So we have multiple faculty learning communities related to active learning, one specifically coming up this year that Lea generously funded from the Active Learning Initiative budget for folks who are looking to engage in SOTL, Scholarship of Teaching and Learning, on their active learning endeavors, as well as specific faculty learning communities for the folks who have peer learning assistance in their courses, or who are engaged in the work after the course redesign experience, for the alumni of the Course Redesign Institute. We also have opportunities for faculty who are highly engaged and master teachers to serve as active learning mentors to their peer faculty. So we have three active learning mentors every year who work with the folks who’ve graduated from the course redesign institutes to give them opportunities for office hours with their faculty peers for micro mentoring sessions on specific topics of interest. They also create supportive resources. We have similar mentors for our PLA faculty, our peer learning assistant faculty. And for those who are serving in those mentoring roles, the Center for Teaching and Learning provides them with opportunities for additional coaching and support on instructional development, educational development, instructional coaching practices that they might bring to bear in those consultative sessions with their peer faculty, too. So the goal here is that we have some workshops that are entry level, some special topics, workshops, the course redesign experiences, some stuff for the master teachers, who are really wanting to be mentors and wanting to kind of serve as role models within their spheres of influence and for others who are moving through this programming, and we’re helping them build capacity in those areas, too. And then, of course, we have faculty on campus who have been working in active learning for years and years and years, and who published on active learning, who are real role models nationally. And so we really try our best to honor their expertise and bring them in to help frame the ways in which we deploy our programming, but also to serve as guest speakers and to engage with folks who are newer to active learning so that we can help provide opportunities for that cross-collaboration and multidisciplinary discussion.

Rebecca: I love the range of options you have available, and really how much you all have embraced the idea of flexibility for your teacher learners, just like we might want in our classrooms as well.

Meg: Yeah, yeah, that’s right.

Rebecca: Can you talk a little bit about some of the active learning techniques that faculty have adopted after going through the institute or other variations?

Meg: Yeah, so as you’ve heard now, our summer institute is three weeks long. And so not only are we giving sessions on active learning, but we’re modeling active learning throughout. And so folks see a lot of different techniques that they might not have seen before. And they’re trying to identify which things that they’re seeing, which instructor moves they see, that align with the needs of their students, of the needs of their course, and their teaching persona as well. So as you might imagine, folks to leave the institute, select different things that are authentic to their practice and their situational factors for their course. So we go into the Institute, which actually begins today for summer 2024. We’re recording on the first day of the Active Learning Summer Institute. So we’re getting the anxious energy of meeting the new faculty. We start out on the first day of helping faculty appreciate that we don’t see active learning as a homogenous set of teaching activities, we’re not going to give them the recipe. We want them to discover the recipe that works for their course. So our QEP is sort of predicated on the notion that the extent to which students are actively engaged in thinking and applying what they’re learning is of far greater importance than the particular active learning instructional strategies used. And so to that end, the course redesign process helps faculty first hone in on the key enduring understandings and learning outcomes at the course level they want their students to leave with and identifying key learning bottlenecks or threshold concepts that they’ve identified that students tend to either have a tough time with, or they’ve really had to work hard to help students grasp that concept in order for them to move from novice to expert in that field. And then we encourage faculty to design their active learning interventions in alignment with those key features of the course so that active learning is more like the trunk of the tree rather than the ornaments on the tree. And our philosophy there is that when pressed for time, the ornaments are easy to cut, but if the act of learning is baked into the backbone or the trunk of the course, that’s just inherent to the identity of the course and it’s not something that’s going to be cut for time. And also you’re putting your energy into where the biggest impact is going to be for your learners..

Leah: If I may add, this was the revolutionary part of being a participant in the Active Learning Summer Institute. I thought I was going for the tips and the tricks, I’d get my little suitcase of ideas and I would just kind of put it all in there through the Institute, and then graduate and use all those as those ornaments as Meg: described it. And instead, I found myself looking at my course and realizing that the backwards design principle had me throwing a lot of content at my students, and not really aligning the activities in the course towards the student learning outcomes. Once I was able to do that hard work, then the excitement of “Ooh, okay, well, which tips and tricks, which kind of active learning engaging styles would be most useful for this.” And that’s just a really exciting way to do it as an instructor.

John: Most instructors, as they come through their educational process, have not been exposed, particularly not in graduate school, to active learning techniques. How do you address issues of faculty resistance to completely changing the way in which they’re approaching the design of their classes.

Leah: It takes a while, [LAUGHTER] having been one of the resistant faculty, I’ll speak to it from this perspective, I think Meg: referenced the former colleague Colleen, and she was my mentor going through this institute. And I remember her asking me to write all the content that I felt was so essential to get the students to to learn throughout the semester, and I distinctly remember this moment where she said, “Okay, let’s put that in an envelope, all that stuff that you just wrote down, we’re just going to put it in an envelope, and we’re just going to set it over here, it’s still very much matters, don’t worry, it’s not going away. It’s all still important.” And then she said, “Let’s really think about the future student. Who are you teaching? What would be a success?” And I think that’s a moment where I could take a deep breath, as a faculty. At first, it was scary and fearful. But it’s a moment for the first time you take a deep breath and go, if I miss a case in international law, they are still going to thrive in this world. I’m focusing on what skills they actually need to thrive in this future career. And so it’s not about that specific case, it’s about the skills. And once I was able to realign that, and it’s so important to be sitting in a cohort, it’s not an isolating experience, you’re sitting in a cohort with other people doing the same reformulating of their whole profession, with mentors from the CTL, helping and encouraging them. Without that sort of ecosystem, it would be a very lonely and harder… just slow… process, maybe, even if you are willing to change. So I think the quality of that instructional development experience within the CTL cannot be replicated. And then I hope the other two kind of prongs of the initiative, once they leave the institute, they’ll still have the mentorship for the next year with CTL and with the faculty mentors. They then can leave and say: “We have the classroom engagement things.” We are also trying to change the environment to help you and we want your feedback. We are always asking those active learning instructors, what can we do differently within these classrooms? And then also the students… we’re engaging the students. So you’re going to walk into class, and they’re going to know what you’re trying to do, and that it’s recognized as a really good thing to do across campus. So it’s the ecosystem that will create more empathy and support and therefore reduce the resistance.

Meg: Yeah, so Leah mentioned, this idea of you have this mentor that exists for the year after the institute. And I don’t know that we explained that. That’s a pretty unique feature of our Course Redesign Institute. And it comes somewhat from our Center for Teaching and Learning has quite a bit of tradition around investment in longitudinal cohort-based programming. So we have a lot of one-year, two-year, multi-month cohorted faculty development experiences, because we find that that interdisciplinary conversation is so helpful for helping folks build connections and shared disciplinary content, and teaching practices with one another in ways that proliferate those great ideas across campus and seed innovation. And as we thought about the ways that our faculty fellows…we call them Faculty Fellows Programs at UGA… ways that our faculty fellows programs were successful, were in that sort of longitudinal support that was available. And we wanted to replicate that as best possible with our Course Redesign Institute. And when we thought about the outcomes of our institute, and how we wanted people to leave the institute, be prepared to deploy a redesigned course in the coming academic year. And we started thinking about what that experiences like as a teacher, and we thought, “Gosh, people are going to leave the institute, be all excited, go back to their department, where they might be one of few who’s engaging in this sort of robust active learning commitment, especially in the early years of the institute.” How can we support them in being successful when they go into their first class and leave the class and go: “Ooh, that didn’t feel like it usually felt I wonder if it went well?” or they do their first robust activity and maybe it doesn’t go quite as they were expecting? How can we be there for them in those moments? So as folks leave our Course Redesign Institute, they’re each paired with a mentor from the CTL. And that mentor is their touch point for the year following the institute, so that when they have that class session that throws them for a loop or they get some feedback from students, that they’re not quite sure how they’re To respond to it, they have a touch point that they can go to. And it’s someone who knows their course intimately, who’s worked with them through the redesign process, is aware of the ways in which their course has changed, and is there to provide them with that support. And also that mentor is the person who performs… we call them mid-semester formative evaluations… but at other CTLs they’re called small group instructional diagnoses. So at the midpoint of their course, someone from the CTL comes in and talks with their students to focus group them, to collect some feedback on how the course is going so far, and then works with the faculty member at closing the loop on that feedback to help students appreciate that they are such an important component of the redesign process too, because the instructor is engaging in this two-way exchange of information with the students both on the content and on the operational organization of the course and refining the course in response to that feedback. So we want the students to appreciate that their experience and the way that they’re learning and what they think could be happening to support their learning even better, how that could be incorporated into the course as it continues.

Rebecca: I really love the depth of the culture change components that are built into this initiative, from feeling like you’re part of something bigger, the fact that it’s voluntary, your support structures, it’s holistic, both Academic Affairs and Student Affairs, there’s opportunities for feedback loops. Change can be really difficult, but when you’re a part of that bigger network, it can be helpful. The timing of this was really interesting when it kicked off…so, a couple years before COVID, so you had some time to establish the roots, and then COVID erupts, AI erupts, there’s more first-gen students, we’ve got a lot going on, and there’s a lot of change happening. How do you help your faculty avoid burnout with so much change and maintain this momentum?

Meg: Yeah, great question. You’re taking me right back to the pandemic and that experience of everyone just doing the best emergency remote teaching you could do. And one of the things that I remember from that period is faculty who had gone through the Active Learning Summer Institute mentioning that they were so glad that they’d gone through the institute because it made pandemic teaching easier. They had recently conceptualized of their course learning outcomes that identified the most important content that they wanted to prioritize, they had a sense of some lesson plans for how to deploy these things. And even though the institute happens in person, because of the way we were talking about how you would make adjustment to different things in different settings, they could more easily conceptualize of how to engage online or asynchronously with students using active learning principles. So we had this early indication that the folks who’d experienced the course redesign process felt better prepared for emergency remote teaching and pivoting quickly. We also found active learning to be perfectly timed as an initiative as a response to post-pandemic disengagement. So when students were coming back into the classroom and members of our community were craving that deep intellectual engagement and community with one another, but then in some instances, were a little bit disappointed when it didn’t materialize in the ways that it might have before the pandemic, we found that faculty were coming to the institute, saying, “This is something that I’m hoping can meet this need that I have to engage students in rich conversation or to engage students in deep intellectual engagement in this learning community, and how can I address this through active learning?” And also, we’re lucky to have a large number of faculty champions for active learning on campus, folks who’ve been using active learning for a while, and in various departments across the institution, not just in specific pockets. And so they’re serving as role models and mentors to the faculty in their unit. And a message that those faculty champions were helping us reinforce is that teaching with active learning is a joyful way to teach. And so oftentimes, when these faculty champions, these mentors, were talking to folks who were saying things like “post-pandemic teaching is a little bit disappointing,” or “I’m feeling a little bit burned out because of everything that we’ve experienced through the pandemic, and through these various disruptions to higher education.” When their mentors were telling them, “Have you considered bringing some joy into your classroom and making teaching a more joyful experience for you and for your students? And guess what? You can do that through active learning. Let me tell you how I brought joy into my classroom through active learning.” We would see folks come to us and say, “I’m just really looking for some rejuvenation and some revitalization, and I’ve seen my colleagues just transition their course through active learning in a way that they’re just skipping down the hall after they leave their class, and I want that feeling. So I’m hoping that this will be that experience for me.” And so we had people coming to us asking for that, and we hoped that would be the case. And then I’ll just add that like many higher education institutions right now we’re in the midst of an evaluation of teaching culture change, too. And so we’re trying to adopt a more what we’re calling a three-voice culture of teaching evaluations that’s relying upon your individual reflection on your teaching some data from your students and the interpretation and contextualization of that data, as well as peer observation of teaching data and bringing that all together as sort of like triangulated data to provide some insights on our teaching, and then respond to that and have like a assessment loop on this. And so it’s this idea of a culture of continuous improvement with teaching where we’re able to proliferate this message of: “You’re not an award winning teacher necessarily on your first day of teaching, but this is a set of skills that you can develop over time. And also, you’re never done developing it.” And so this culture is helping to amplify the message that taking part in educational development is not just an expected part of being a teacher, but it’s an opportunity to continue your culture of continued mastery.

John: So one of the ways in which this is spread has been from faculty to faculty. One of the things I’ve been noticing a lot on my campus is that it also sometimes spreads from students to faculty. As more and more faculty move from faculty-centered teaching to student-centered teaching, students become a little bit more vocal sometimes about expressing what works for them and that will often result in faculty considering using new techniques that their students have recommended to them. Has that been happening as well at your institution?

Leah: That’s why we wanted to capitalize on this proactively and have channels of growth. And so again, we have little areas in which we have students really playing that intentional role. So rather than maybe a complaining, “I went into this class and I didn’t like it” kind of a customer satisfaction problem area, we really want to make sure it’s more that students, first of all, are trained in the active learning pedagogy. So my active learning ambassadors, as they are chosen, we then say, “Okay, here’s the wizard behind the curtain friends, come join us [LAUGHTER] and see what’s going on behind these intentional classrooms that you really enjoy and don’t quite know why you like it so much.” So we bring them into the understanding of what instructors are doing when they’re in an active learning classroom and why. And then we give them opportunities to then go and front face and tell other faculty, “this is what an active learning classroom feels like for a student.” This is why one of my favorite moments… we have, again, as I mentioned, an active learning summit every year in February, and there was an entire breakout session run by students who were peer learning assistants and active learning ambassadors. And they were talking about the study skills that they have seen through their training, or through their classes that really work in their discipline that they use outside of the classroom to kind of empower themselves to study and to learn. And what was wonderful is that the front of the room were students in a panel and most of the room or faculty and I found that the question and answer period, the instructors would raise their hand and say, “Okay, well, but I’m not in that discipline, but I’m thinking about bringing this into my class in X way, is this something that you would find effective?” And they were asking the students about designing their course to be student centered. And that’s when I think if we put those students that know what is going on, so it’s not a complaint section, it is more a reframing and a sharing out of we know what you’re trying to do and we really appreciate it, and here’s how it works really well, for us. It just starts to kind of change that conversation from being yet another instructor versus student dynamic, which I think we have plenty of those around campuses at any given time to becoming an instructor and student experience.

Rebecca: Having student ambassadors and student voices celebrating this work, is certainly a testament to the success of this work. And you’ve also mentioned some other moments of assessment. Can you talk a little bit about how you’re assessing this initiative?

Leah: There is a rich assessment team who is actually part of the larger Active Learning Initiative. And so I hope I do justice to their very nuanced framework that they have. We have macro level, mezzo level, and micro level indicators that we are trying to capture. At the most macro level demand is one of the earliest indicators of success for us. Are people applying to the Summer Institute? Are ambassadors applying for the program? How many peer learning assistants apply and how many instructors want them, just really kind of those basic quantifiable amounts for demand. And then we also are attempting to capture survey data across the university. We stop the students and say: “We don’t know if you’ve interacted with active learning in the classroom or not, we’re not selecting a given class, we’re just wondering what you would say if you observe a cultural change going on.” So we’re also trying to get these larger macro level indicators. Then at the mezzo level, we’re getting into those lifelong learning dispositions that Meg: referenced earlier, the curiosity, the initiative, the reflection and connection. We have artifacts that we pull from courses that have an instructor who’s either going through all the Active Learning Summer Institute, or who has gone through it already, so we do a pre- and post-analysis. And we have faculty scorers, who can go through these artifacts to see if we are capturing if the artifacts include a prompt about curiosity or a prompt about initiative. And then we are attempting to measure, and we’re working on different ways to do that, but measure if we’re seeing growth over time in these lifelong learning dispositions, and then finally, at kind of the micro level, we’re making sure that the disciplinary knowledge is being enhanced within this that are focused on active learning techniques. One thing we have yet to mention is yet another part of our holistic umbrella. This is why the programming is so large, but we have these active learning change grants. And this felt really good to hear for me as an instructor who went through the institute. And then it was about five years later that the next person in my department went through the institute. What these change grants do is it really allows those who go to the institute to go back to their department or their unit and say, “What if we change several classes at a time through some sort of infusion of deep learning and the CTO will support us, the assessment team will help us assess whether we’re doing what we’re saying we’re doing.” And so on this micro level, the assessment team is really starting to capture some pretty cool indicators, that when you kind of create a unit change, there’s a cascading norm it’s called in international affairs, where you change enough of the unit and enough of the actors in a given scenario, you can then really see that norm start to shift in a unit. And so at each of these levels we’re finding it’s still early. So we’re not saying “It’s a win, close up shop,” [LAUGHTER] but we are finding really robust indicators of early success.

John: Well, you’ve given us and our listeners a lot to think about, and it sounds like a model program that could be copied very productively at other institutions. But we always end by asking: “What’s next?”

Meg: We’ve started to hear from folks that are interested in proliferating initiatives. So, of course, we welcome conversations with folks who are interested in hearing about some of the elements of the program and brainstorming ways to deploy these on other campuses, lessons learned along the way, etc. I mentioned earlier that as folks crave these multiple on ramps, we’re deploying these additional on ramps over time. So starting in the fall, we’ve designed some of our foundational active learning workshops, we’ve taken those and created asynchronous online versions of those workshops that are going to be hosted in our learning management system. So they’ll be available on demand. So if a faculty member doesn’t happen to be available at the time that we’ve offered that workshop this semester, they can still engage, especially in those foundational workshops. And then the spring semester course redesign experience version of our Course Redesign Institute will be deploying for the first time in spring 2025. We’re really excited about that as well.

Leah: I would also say for your listeners, if there is an interest in kind of seeing what we’re up to and helping participate in engaging with this discussion, traditionally the active learning summit has been really focused on UGA community for the first two years, but in the third year, so February 2025, we are going to open up the summit to visitors and encourage them to be a part of these breakout sessions or propose different ideas that they would like to talk about. I know Rebecca mentioned earlier, the challenges of AI, we’re going to try to have a breakout session, though, that focuses on this: active learning best practices in the age of such change. We’ll send out a call for applications in the fall, and we’d love to have anybody who’s interested in learning more, I think it will be Thursday, February 27, and Friday, February 28, on UGA campus, and we welcome all those who are interested in attending.

Rebecca: That’s great. Thank you so much for joining us. There’s so much to think about and so much exciting work that you’ve all done.

Leah: Thank you.

Meg: Thank you so much for having us.

John: Thank you and we look forward to more conversations in the future.

Meg: Absolutely. Anytime.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

 

347. CATs and AI

Classroom assessment techniques, initially developed at a time when chalk-and-talk instruction was the norm, helped to shift the focus from teacher-centered to learner-centered instruction. In this episode, Todd Zakrajsek joins us to discuss how generative AI can enhance these techniques by providing more immediate feedback.

Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published five books (so far) in the past five years. His most recent book is a 3rd edition of Classroom Assessment Techniques: A Handbook for College Teachers, with Thomas A. Angelo.

Show Notes

Transcript

John: Classroom assessment techniques, initially developed at a time when chalk-and-talk instruction was the norm, helped to shift the focus from teacher-centered to learner-centered instruction. In this episode, we explore how generative AI can enhance these techniques by providing more immediate feedback.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Todd Zakrajsek. Todd is an Associate Research Professor and Associate Director of a Faculty Development Fellowship at the University of North Carolina at Chapel Hill. He is also the director of four Lilly conferences on evidence-based teaching and learning. Todd is the author of many superb books, and has published five books (so far) in the past five years. His most recent book is a 3rd edition of Classroom Assessment Techniques: A Handbook for College Teachers, with Thomas A. Angelo. Welcome back, Todd.

Todd: Thank you, Rebecca.

John: It’s nice talking to you again. It’s been a while. I think the last time was last summer.

Todd: Really? It seems like yesterday. Time goes by. It was last summer. You came to my office and recorded.

John: That’s right.

Todd: It was fun. You left me a really nice set of tea. Wooh.

John: And a mug.

Todd: Yes, I have a mug for the show. [LAUGHTER]

John: Speaking of teas, [LAUGHTER] today’s teas are:… Todd, are you drinking tea?

Todd: Every time I’ve been on this program I’ve been drinking tea and I’ve had some pretty exotic brands and blends but, I’m sorry, today I have water, the lightest of all teas.

Rebecca: It’s a very light tea. I have English Tea Time tea.

John: I have Lady Grey today.

Rebecca: That’s a good choice.

Todd: Alright, I think I came in third place.

John: We’ve invited you here today to discuss classroom assessment techniques. The first two editions to this book were written by Thomas de Angelo and K. Patricia Cross. In last week’s podcast, Tom joined us to discuss the origin of this book, where you joined as a contributor for the current third edition of this classic book. What was your reaction when you were first asked to join in this project?

Todd: Well, this is actually kind of a funny one, because it goes back to Linda Nilson and Teaching at its Best. And Linda was retiring, and the publisher, Jossey-Bass wanted to do a new edition of Teaching at its Best, and Linda says “Why don’t you see if you get Todd to do it and I will work with him on it and do it.” And we did it. And we got the book done a little bit early and on schedule, and it came out very, very well. So there’s a little plug for that book there. And it must have been about two months later, I had called them and said: “If you find another project like this, I love working with people like this.” And a couple of months later, I got a phone call and they said “We have another project we’d like to potentially have you work on” and I said: “Great, what is it?”… and they said: “Have you ever heard of the book Classroom Assessment Techniques?” And I held it together, and I said, “Well, yeah, yes, I have.” And they said, “Well, we thought it’d be helpful given what you do and what Tom does and everything, is maybe the two of you could work together on this book.” And so he talked a little bit about what there was and everything, and I hung up the phone and did a little happy dance because this book was one of the most influential books in my entire career. But the way the universe works is I ended up doing this one because of the work with Linda Nilson’s. And so that’s how I got started on this.

Rebecca: That’s amazing.

John: And, I agree, that is an excellent book as well. So you’ve got two classic books that you’ve worked on recently.

Todd: It’s really surreal, you know, a first-generation college student and every once awhile is kind of like I just can’t believe this is happening. But it was a really good project.

Rebecca: When did you first start using classroom assessment techniques in your classes? You mentioned that this book was influential.

Todd: I taught my very first course in 1987-1988. This book came out, second edition, 1993. And I was using it pretty much the moment it came out. Somebody had pointed it out to me and I didn’t know anything about classroom assessment techniques, and then I started reading through it and picking some out. I thought it was genius. Much like periodically, what you’ll do is you look at something and think this is ridiculous that nobody started doing this before. In some respects, it’s not that hard, right? You ask people questions, and you find out if they’re learning. It just took Tom and Pat to come along and say, “Why don’t we come up with strategies to find out if they’re actually learning.” So I think it was really influential for me because it kind of struck that chord with me that I always believed we should be finding out how much students were learning instead of focusing on teaching. But that’s not what was going on at the time, everybody was focused on teaching-centered educational practices.

John: What do you find most valuable about these classroom assessment techniques?

Todd: It’s interesting, because there’s a couple different levels. The one that’s really valuable, of course, is you find out if students are learning. Secondarily, you can find out if you’re teaching well, and you can change your teaching practices based on what they’re learning and based on what’s going on. But I’ll tell you the real reason I think this is really, really valuable is it addresses equity issues very, very cleanly. We walk into classrooms with all kinds of preconceived notions and stereotypes and implicit biases. And it’s really, really easy to think that a student’s struggling for a certain reason, unless you ask them. And so, for several of these strategies, I think it’s great because I’ve had students that have been practically sleeping through every class and then I started doing the classroom assessment techniques and he had some of the best responses, this one student did. And I would just totally blown away that this person was just processing it to such a level and I had a preconceived notion based on where the person sat and how they dressed and how they acted in class. I didn’t think they knew anything or they didn’t think they were learning and they really were. So I think it’s huge for equity and overcoming some of the biases we have in our classrooms.

Rebecca: That’s a really good example to underscore the importance of that. We had the opportunity to ask Tom what his favorite classroom assessment techniques are. So I’m really looking forward to asking you the same question: what are your favorites?

Todd: I’m one who says over and over again, in education, that we should always be careful about absolutes. Like, there’s never a strategy that always works. There’s never a type of student that’s always the best. Any situation can change. So there’s like 55 CATs in there. And I listened to the program with Tom and I heard him say that he didn’t have his favorites. And then he quickly picked out his favorites, and I’m sure it’s the same with his daughters. [LAUGHTER] So I have three girls, you know, they changed the favorites, I would definitely go with the standards of the muddiest point minute paper, those two are the big ones. Everybody’s done a minute paper, probably at some point. The muddiest point I like, which was the adaptation. And if you haven’t heard this show with Tom, make sure you listen to that, because he explains that very well. I think I like exam wrappers where you go in and kind of have the students talk about the exam experience. But for me, there’s several CATs that are based specifically on assessments. And it’s the thing that we almost never ask students for, and it’s the thing they can give us the best feedback on. So it’s like, when you have group activities, it’s an assessment on how well the students perceived that the activity worked for them. So instead of me just saying, we did a great group project today, you ask the students: “Was this a good group project?” There’s assessments in there about learning interests, what types of things you’re interested in, your study strategies, exam strategy. So you can ask students those things. And I think those make great classroom assessment techniques, to find out the processes that you’re doing, and how the students are receiving those.

Rebecca: Those are good opportunities to respond in the moment in your class, but also probably plan in the future too, to know what to do next time around, perhaps.

Todd: Yeah, I think specifically for exams. And one of the favorite questions I’ve always had on exams, because the students would get to a point where, and I got this, because of one of the assessment techniques I’d use early on, is students always were frustrated that there was something on the test that they studied really, really hard for, and it wasn’t on there. The problem was, it wasn’t always the same. So the students all had different things they studied. So because of collecting that information, and finding out from the students as they were frustrated, I started adding a new question at the end of every exam worth the same as some of the other questions there, was: “Please describe in detail something that you study that I didn’t ask you about on this test.” And students immediately felt better about the exams, they were happier with the test, because they didn’t spend four hours memorizing something that they perceived wasn’t valued on the test, even though the real reason for all of this stuff is life, not the test. But again, that was one from just asking the students: “What do you like and not like about the test?”

John: That’s a really great application of a technique that I hadn’t thought about, I can see how students would feel that their voices are being heard with that type of thing.

Rebecca: …or that their efforts were validated.

Todd: They got their points for their studying, and it just really seriously changed the affect of the class.

John: In the last podcast we did with you last summer, we talked a little bit about how teaching has changed from the time when you and I both started teaching, and this book was written back around the time when you and I were both relatively new at this. What role do you think this book has played in that evolution of instructional practices?

Todd: Well, first of all, I do want to acknowledge and help people just really understand the difference between 1993 and 2023. Some of the things we took out of the book was “display your results on an overhead projector,” there was no mention of the internet anywhere in that book, because the internet wasn’t here yet. Just stop and think about that for a second. So it was a very different time. A lot of heavy lectures. I mean, a lot of times people were lecturing, there were other activities, or other strategies, there was group work and role plays and those types of things. But primarily people are lecturing, because that’s just what people did, primarily. And so the role that this played, I would say, I don’t know if I’ve ever said this before. I think this book had a bigger influence on higher education than any other single book that’s out that I know of. There are some really, really good books out there. But this fundamentally changed the game. Because what it did was it started asking students about their learning, which means it started to really shift the focus from teaching to learning. This did happen to come along, by the way, if you really look at dates, the whole concept of really pushing active learning in higher education, at least in the United States, really took off in the mid- and late-1990s. So 1996 to 1998, we started seeing people like Richard Hake, do some huge studies about this. American Association of Higher Education was changing their themes from taking teaching seriously to taking learning seriously, all of those things happened about three to five years after this book came out. So I think this was the impetus for a huge, huge aspect of the whole push for moving from teaching to learning. I think it had that big of an impact.

Rebecca: In recent months ChatGPT has hit the news everywhere. We’ve heard all kinds of reports in higher education journals and things. accreditation organizations are talking about AI. And many faculty members are also talking about potential harms of AI or other see benefits. What do you think the impact of AI tools will be on higher education practices? …speaking of evolution… [LAUGHTER]

Todd: I don’t have the statistic right in front of me, but there was one I did a presentation just recently on how long it took to get to a million users. And they were talking about some things like Netflix and other things of taking two and a half years, and then six months and stuff. And I believe ChatGPT was five days. So it’s pretty incredible. So Rebecca, I love the question. I think this is a huge issue. The two biggest things that I heard out of the gate was: number one is how are we going to ever have students do their own work again? It was all academic integrity all over the place. And I couldn’t help but talk to a couple of people who were really freaking out about it and I said, “You should really read some of the research about academic integrity already and the proportion of students who are doing things that they shouldn’t be doing, so to speak, there.” And so that was it… academic integrity. And number two was like, “What kind of assignments should we do now? How do we structure assignments to be meaningful?” And I find this fascinating, because they weren’t talking about the learning process. And I still, to this very moment, I’m just starting to hear it now, but I have not heard much of anything about how can we use generative AI to influence the learning process in a positive way. And so I think that’s the huge thing we need to be looking at. So I guess the people who do know me, know me as the individual who writes a lot about helping students to learn and teaching tips in the classrooms kind of things. I think from a cognitive psychologist’s point of view, we can look at some of the foundational things like: we know that repetition is really good, we know that the process of explaining something really helps because it’s a chance for you to practice at retrieving information. We know spaced recall is really good. And there’s all these things out there that can kind of do that. But I’ll tell you, and I’m coming right back to the CATs, but ChatGPT and some of the other generative AI has some phenomenal opportunities. We know that teaching this is good. So I opened up by ChatGPT 4 the other day, and I was practicing and I typed in: I need to learn about metacognition, would it be okay if I taught you metacognition, and then you tell me how good of a teacher I am. And the chatGPT says, “Yeah, this will be fun. So let’s get started.” And I said, “Okay,” and so I explained metacognition. And ChatGPT asked me some questions like, “Well, what does this mean? And what does that mean? …and I acted just like I was teaching somebody. And then when I was all done, I said, “If I were a teacher, right now, how would you grade me?” And I did not get a good grade [LAUGHTER] which was kind of disconcerting there. But I said, “Why did I not get a good grade on this? And then it explained a section that I had completely forgotten, because I was doing this off the top of my head to try it. So I went back and explained that. And then ChatGPT says, “That’s much better.” I think that’s phenomenal. Another one, I will not get into details, but I typed in, or I gave it the prompt… it’s all about the prompts folks… “That I just learned in my introductory psychology class at a medium-sized, midwestern university. I’m a C-level student who’s worked really hard, and what I just learned were about persuasive techniques. Could we pretend we’re on a car lot, and I’m a salesperson, and you’re going to buy a new car, and you think you want to buy it today, but you’re not going to rush into anything.” And ChatGPT says, “This sounds like fun. Tell me about your best selling model.” [LAUGHTER] And I proceeded to explain all these things. And the scariest one is I said, “This Ford van right here has excellent ratings for safety.” And it said, “That’s particularly important for someone with a loved one and two furry friends.” So the learning possibilities are all over the place for this thing. And so for classroom assessment techniques, this is great, because we can actually teach students some strategies, how in the classroom, how to assess whether they’re learning. We can replicate these and have them do them with AI. So we could use AI to do things we couldn’t do before. But we can also do it to model things for students that they can do on their own. So I think there’s some really, really interesting possibilities coming along.

John: And one interesting one is if you ask ChatGPT for your biography, you may very well turn out to be a car dealer in the Midwest.

Todd: You know what, I’m just gonna jump in there and say that is totally true when ChatGPT was about six months old, because I did it. I said, “Write a letter of recommendation for Todd Zakrajsek… and you probably saw this, why you even mentioned it… and I think it had 19 things I counted in there and this term of hallucinations, but 17 of the 19 things were wrong, including where I went to school where I worked and everything else. That was when ChatGPT was a baby. I asked it the same thing just recently, and it was scary, how close it just nailed everything. But yeah, it was for a while.

John: It’s improved quite a bit, but one of the nice things about it is our students come in with really diverse backgrounds and we can provide supportive materials for students when they have some gaps in their prior learning. But, ChatGPT offers the possibility of having something that’s completely personalized to their needs, so that they can ask questions that are specific to the issues that they’re having trouble with, and not what we might have guessed they’d be having trouble with. So, it’s something I’ve encouraged my students to use in my intro economics course, and they appreciated the fact that they were encouraged to use it,

Todd: I think that individualized instruction is going to take on a whole new level. And so I think that’s going to go crazy. But I will say this, and I think it’s extremely important. I’ve been in higher education for 40 years, I have never seen a bigger possibility for an equity gap to be the widest chasm it could ever be. I think we got equity problems that are down the road. And if we don’t pay attention, it’s going to be awful, because I believe that generative AI is going to be a huge learning aid, and it’s going to help us with lots of things, for the people who have it available to them, they will skyrocket and be able to do stuff; the people who don’t have it available… reliable internet, a good computer, or some kind of a device, a safe place to sleep at night… those individuals are not going to be on that tract and they’re gonna get separated. Right now, there’s a couple of grade differences between people who are privileged and not in some classes. That could widen so much. So I think, everybody, we just got to really be careful about how to make sure that this was as equitable as possible for everybody.

Rebecca: Higher ed leadership at various colleges and universities are certainly dancing around the idea of AI and what kinds of policies and procedures to put into place. What would you advocate for? What would you encourage faculty members to advocate for to really make sure that equity is addressed?

Todd: I think one of the very first things we need to do is teach students how to use it properly. Because one thing we know about learning is, the more you know about something, the easier it is to learn something related to that topic, whatever the thing is, and what that means is that learning is not linear, it’s curvilinear. The more you know, the easier it is to learn something, which means you’re going to know more quickly, and as you know more quickly, you’re going to be able to learn more quickly. And this is why people struggle at the beginning with things and then all of a sudden they take off. So if some students know how to use the systems, and some don’t, that’s where we’re going to run into some huge issues. So I think one of the things we can do is teach students how to use it to help you, and when to be careful that it’s not going to help you. The movie Wall-E is a great thing to be keeping in mind. If you haven’t seen the movie Wall-E, it’s a plug for the movie Wall-E, it came out a long time ago, but that concept of the humans that had everything done for them ended up becoming just blobs that laid in chairs all the time, and I think our brains basically do that. If students use AI to do their work, they’re not going to develop critical thinking skills. If students use AI to help them to practice at retrieval and spaced out practice and do those things, they could become very good. So I think schools need to be careful about helping students to understand how this can really help them. That’s the first thing. Then there’s the obvious, helping people to have access to technology, we could have labs open, we could have laptops that we can loan to people, reliable internet, we could have rooms that we keep open 24 hours a day, keep them 24/7 there. Some libraries during exam time, they don’t close for like four or five days. We could have rooms in the library that people can come and have safe spaces to study and work, reliable transportation is going to be an issue, we’re just gonna have to work out how do we address those issues. It’s gonna be a challenge, but we need to be thinking about it.

John: The use of classroom assessment techniques has been growing steadily since they were first introduced. And each time there’s been new educational technology, it seems to have led to an increased use of those. During that time, we’ve seen the introduction of relatively low-cost computers, computer networks, the internet, mobile computers, and so forth. How do you think the availability of generative AI will affect the use and value of classroom assessment techniques.

Todd: I think there’s a couple things that’s going to really, really change. I think for classroom assessment techniques, number one is we’re going to be able to individualize a little bit more. So we can tailor… we don’t have to ask the same question of all the students, we don’t have to look for all the same type of responses. So we can think about it a little more creatively in how we can use it, almost like an individualized instruction toward classroom assessment. Probably the biggest thing, though, overall, is it’s going to allow us to just process data in numbers and levels we’ve never seen before, and particularly free responses. In the past, we’ve often ask closed-ended questions just because if you’ve got a class with 400 students, you can say: “On a scale of one to five, to what extent did today’s lecture help you to understand something?” And the students can pick a number, we could drop that into an excel sheet, or anything, and come up with a number very quickly. We can now say on a muddiest point, “What are you still struggling with?” Take 400 responses, dump them into a generative AI program and have it spit out five things within 30 seconds:” Here are the five things your students are struggling with.” So I think it’s going to allow us to do more qualitative types of things very quickly. And I totally get that this is not hardcore qualitative research with good analysis of the data. But for what I need in the classroom, I think we’re gonna be able to get those responses very quickly and in real time. So what that means is, we’re actually going to be able to ask something like a muddiest point at halfway through the class, and then I could have the students do a quick think-pair-share, while I analyze the information and a matter of three minutes later, they come out of the think-pair-share, to talk a little bit about what they talked about. And then I could say, I see that you’re still struggling with these concepts, so let’s revisit these things before they ever leave the room. That’s never been possible before.

John: When I’ve been teaching classes of two to 400 students, I’ve used the free response option and word cloud. But the word cloud is just highlighting individual words, the ability to do analysis in more detail is going to be incredible. And I think most providers of response software are working on introducing AI, and some of it is expected to be available fairly soon.

Todd: Oh, there’s some great stuff come check GPT and some terribly scary stuff at the same time. By the way before I forget this, because I have to tell you this because I have ADD, that’s one of my favorite phrases.ChatGPT 5,, one of the things I read about that’s going to be coming up fairly soon is the ability for it to launch his own AI’s as needed. That’s the one that we’ve been waiting for, and thinking, “Hmmm, that’ll be interesting.” But yeah, I think that the ability to read through and look through the information is just going to be a game changer. And online synchronous/asynchronous, face-to-face in all environments. We could look at stuff in asynchronous environments in ways we haven’t before as well. I could look at different ways that students are responding, I could ask classroom assessment techniques. In fact, they’re in the book. We have several of them… ways to use CATs in asynchronous environments, and AI is changing how we’re doing it.

Rebecca: Can you give an example of how that’s happening in asynchronous environments?

Todd: In terms of asynchronous environments, everybody likes to go to discussion boards, but quite frankly, discussion boards are boring. We could have students generate something like the script for a commercial. And after they do that, I could have the CAT that comes in to say, “To what extent was this helpful in the learning process for you?” And the student, as they’re developing this thing and as submitting it they could also submit, right along with it, their CAT that comes with it, and then I could be reading the CATs as they come in one by one right behind each one of them. And so the concept, there, would be kind of a real time CAT analysis that’s not waiting until there’s a whole group and then looking at the group. Because typically, when we think about these CATs, it’s like, the 400 students fill out a muddiest point and we analyze the 400 students. We don’t need to do that. In an asynchronous environment, we can have these CATs coming in and we can be analyzing them as they come in. And we could even add them to the previous ones, to group them if we wanted to. But the bigger one is, I can find out how the students are doing as they’re doing. So that’s probably one way to look at it.

Rebecca: One of the things that you mentioned in an in-person or a synchronous setting was being able to analyze results mid class. And that’s in part because a lot of our students have mobile devices and technology that they bring with them that they haven’t had before. Do you see other ways that these mobile devices are also changing the way instructors might implement CATs?

Todd: The way you just mentioned, though, I can’t think of many other than the concept of as an information gathering device. I will say, just because I happened to read this study just a little bit ago here, the devices are very interesting, in a sense that we’ve seen high schoolers tend to have the same amount of time they spent with their peers was very consistent until about 2012. And in 2012, the numbers started dropping off pretty quickly. And then COVID dropped it even faster, but the slope was already there. And the point here is that people are turning their attention to devices so much that they’re not talking to each other exhibited by when you see two people at a restaurant having dinner, and each one of them are on their device. There’s all kinds of psychology about how those things happen. But the overarching thing that’s the issue is that we’re having students at levels we’ve never seen before disengaged in the classroom, even when we’re trying to do CATs and engage. And part of that is because of the devices. So one of the things that’s kind of interesting about this is when you say: “Okay, everybody gets your devices out, and now we’re going to use them for a CAT.” And then we stick with that and how we use that data makes a big difference of whether they stick with it or go back to what they were doing. Strategically, it’s really helpful to use those devices to engage the students.

John: One concern with generative AI is many of the types of assessments we’ve used before can be answered very nicely by generative AI. What might we do to reduce the likelihood that students will use generative AI as a substitute for learning?

Todd: I think it’s really, really important to talk to students about the long-term implications. I know this is not going to be for everybody. I totally get that. But one thing I’d say to students is: “I don’t understand why anybody would run, why would anybody go jogging? You can just get in your car and get there faster.” So if you kind of pitch that to them, that idea is for most of the students, if there are any kind of health-related fields, or if they want to have a cardiac system later in life, exercise is just one of the most important things you can do for your body, by the way, in terms of that getting 150 minutes per week, and just higher respiration. If you take that away, it is bad for the human body. And so the response is the same thing is true of this. Just talking to the students. If you use generative AI to answer your homework problems, to develop the quizzes, to write a poem, and I’ve done all of these in workshops, where I’ve said to people, I can do this real quickly. I don’t care who’s in this room, I can ask you to write a short story. And I will crush you with generative AI. The problem is that, if I do that, I don’t learn how to think for myself. And so I think the biggest thing we can do for students, number one is to build community in the classroom, to number two, to tell them what this all means, and then number three, ask them, ask them, how it’s helping them, ask them how it’s hurting them, how they believe these things are working. And those are classroom assessment techniques, we can use these assessment techniques to find out to the extent that they’re doing this. While I’m on my little diatribe, we will never be at this spot again in our lives. We have just developed something that can totally short circuit cognitive processing and critical thinking. But we did it with people who have critical thinking skills and cognitive processing. So we developed something because earlier in our lives, we had to learn in the ways of joggers, we had to develop our systems. And now we can talk about the automobiles that would move as faster. The students coming along are going to step right into those automobiles, right into the AI, and if they’re never told that it’s important to go out jogging, they’re not going to develop the systems that are going to be needed later. And so again, just think about this for a second, we’ve developed a system using a system that may disappear if we’re not careful. And so I think that’s where we can change it. So it boils down to community, why we’re using it, and then assessment questions of how it can be used, and where it’s good and bad.

Rebecca: I think students really enjoy having conversations about AI and exploring how it’s useful and not. I have had activities in my class where we intentionally used AI to see what it was like and when it would be useful and also analyze where maybe they tried to use it and it was totally not helpful, and why it wasn’t. And they really appreciated those kinds of conversations and learned a lot from those.

Todd: I think the students do. And we could pick different spots in history. You go back to Socrates and his whole belief that if you write things down, it weakens the mind. It makes sense. If you have to learn it, you’re going to be much stronger than if you write it down. But I can’t imagine right now teaching without students writing things down. And Samuel Johnson came along several 100 years ago and said, “There’s ready availability of books. With books all around, why would we really need to teach this stuff, they can just go get a book.” I can’t imagine teaching without books right now. So these things that everybody got scared about, or thought that just going to change just became integrated. And when the internet came about, we talked about this a while back, in the sense that many of us were teaching before the internet actually showed up. But when it showed up the people who were teaching were freaked out. “How am I supposed to teach when the students can go and get anything they want off their computer?” And now I can’t imagine teaching without the internet. I believe five years from now people are going to say I can’t imagine teaching without writing things down, without using some kinds of print format, without using the internet, and without using generative AI. I don’t even know how I would teach without it. What that’s going to need is the same stuff we’ve done in the past, that’s how do you teach well with the Internet? How do you teach well with generative AI?

John: It’s an exciting time to be in the midst of all these changes. And it’s going to be interesting to see how we all answer those questions as we move forward.

Todd: It is, but before we wrap up, because I can tell that tone, [LAUGHTER] I want to put a general call out that there are several people who are saying: “I wish I could retire right now” or “This is a great time to retire.” The statement I really want to make is this is a hideous time to retire if you’re really good at what you’re doing. Because we have never needed humans with really good critical thinking skills as much as we do now, maybe we will later, but to date I don’t believe we ever have. And for some of the people who are saying “I just don’t want to teach anymore. This is awful…” we might need you more than we’ve ever needed you, so I don’t like this concept of the mass exodus of certain people.

Rebecca: It’s an important point.

John: One other thing I will add is Paul Samuelson once said, in describing the way in which models of the economy evolved, that “funeral by funeral, the science makes progress.” So there is the counter argument there too, that people who are very tied into the old ways of teaching do need to either adapt or perhaps they’ll be replaced by people who are more willing to try new alternatives.

Todd: Do I smell a little Kuhn in there? The revolution of science. Ah yes.

Rebecca: And we always wrap up by asking: “What’s next?”

Todd: I’m trying to keep busy. I’m kind of on a roll right now. And I want to tell you, I’ve been writing books and the only reason I’m unhappy about ChatGPT and gen-AI come along is people are going to assume that I’m using them so that I can crank stuff out. But thus far, I have not used any of this generative AI to write any of these last books that I’ve done, these five books in the last five years or something, and enjoyed working with Tom on this one. But I got another one coming out in about four months. I just turned in the final pages for that And it’s the Essentials of the New Science of Learning: the power of learning in harmony with your brain. That book should be out in a couple of months. And so the what’s next after that, I am working on a book right now for helping with neurodivergent learners. We’ve joked around in the past, I’ve got ADHD is about as bad as it gets. If it were a competition, I could come in probably close to first place, and also on the autism spectrum just a little bit too. And when I started sitting down with some of my colleagues who also have ADHD and autism spectrum, it occurred to me, we don’t talk to students enough about these things. And I think the challenge of that is that the students believe that they’re alone, and they’re not sure they can do things. And I was talking to a student fairly recently that said, “I now believe I have a shot, here you are with a PhD, you’ve written books, if you’ve got ADHD, as bad as you claim you do, maybe I can do something.” And then that’s when I really really was getting serious about we need to help folks out. And the topic I’ve been playing around with a little bit in a couple of workshops is The Ones Too Often Left Behind as the title I’m using. And it’s the students that aren’t built for the system that we developed. And so I think that we need to treat some folks out there a little differently. And I think we can really build up the pool of intelligent folks by helping to teach the people who just learn in a different way.

Rebecca: It’s really important work to have models and put models out in front of students because they need to see themselves in whatever discipline, field, etc., that they want to pursue. I’m excited to hear more about that.

John: And when will these books be out? You mentioned the timeframe for one of them.

Todd: The Essentials for the New Science of Learning should be out in September. And then the other one, if everything goes well, is probably looking at a February date that it would probably be available.

Rebecca: Well, it sounds like we’ll be talking to you soon then, Todd.

Todd: We may be. When you got ADHD pretty bad, you just can’t really predict when that books gonna be. But yeah, I’d love to chat with you when it does finally emerge.

John: And what are you going to do next week?

Todd: You know what? You gave me a great idea. I think we should do a book on procrastinating people with ADHD. [LAUGHTER]

John: It’s always great talking to you. And we’re looking forward to each of these projects coming to fruition and we will be talking to you about each of them, we’re hoping.

Todd: I appreciate the opportunity to come and chat with you. I think you two do a phenomenal job with your interviews and the programs that you pull together on really good topics and I just am honored to be one of the guests.

John: Thank you.

Rebecca: Yeah, thanks. We always enjoy talking to you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC] </p

346. Classroom Assessment Techniques

Classroom assessment techniques can be used to shape instruction to the needs of our students. In this episode, Thomas A. Angelo joins us to discuss the origin of these techniques and evidence concerning their efficacy.

Tom is Clinical Professor Emeritus and Director Emeritus at the UNC Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill. In addition to teaching for 40 years, he has been long involved in professional development and has served as faculty member and Director of teaching, learning and assessment centers at UNC, LaTrobe University in Melbourne, Australia, Victoria University of Wellington, New Zealand, the University of Akron, and Boston College. Tom is best known for his work with K. Patricia Cross on Classroom Assessment Techniques: A Handbook for College Teachers, which was first published in 1988, with a second edition in 1993.

Show Notes

  • Cross, K. P., & Angelo, T. A. (1988). Classroom Assessment Techniques. A Handbook for Faculty. Jossey Bass Wiley.
  • Angelo, T. A., & Cross, K. P. (1993). 2nd. Ed. Classroom assessment techniques. Jossey Bass Wiley.
  • Angelo, T.A. and Zakrajsek, T. (2024). 3rd. Ed. Classroom Assessment Techniques: Formative Feedback Tools for College and University Teachers. Jossey Bass Wiley.
  • Schwartz, Charles (1991). “An Academic Adventure.” March 4. – A description of the origin of the minute paper by it’s originator.
  • Chizmar, J. F., & Ostrosky, A. L. (1998). The One-Minute Paper: Some Empirical Findings. Journal of Economic Education, 29(1), 3–10. (the article that John referenced as his first reference to this topic)
  • Mazur, E. (1997). Peer instruction: A user’s manual. Pearson
  • Mazur, E. (2014). “Confessions of a Converted Lecturer” – a recording of a presentation by Eric Mazur at SUNY Oswego, 5/19/14
  • Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. John Wiley & Sons.
  • Pam Hook and Josie Roberts (2018). “The who, what, when, where, and why of SOLO taxonomy.”
  • Angelo, T.A. (1993). “A Teacher’s Dozen—Fourteen General Research-Based Principles for Improving Higher Learning.
  • Pre-order for 3rd edition of Classroom Assessment Techniques: Formative Feedback Tools for College and University Teachers.

Transcript

John: Classroom assessment techniques can be used to shape instruction to the needs of our students. In this episode, we discuss the origin of these techniques and evidence concerning their efficacy.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Thomas A. Angelo. Tom is Clinical Professor Emeritus and Director Emeritus at the UNC Eshelman School of Pharmacy at the University of North Carolina at Chapel Hill. In addition to teaching for 40 years, he has been long involved in professional development and has served as faculty member and Director of teaching, learning and assessment centers at UNC, LaTrobe University in Melbourne, Australia, Victoria University of Wellington, New Zealand, the University of Akron, and Boston College. Tom is best known for his work with K. Patricia Cross on Classroom Assessment Techniques: A Handbook for College Teachers, which was first published in 1988, with a second edition in 1993. Even if you’ve never heard of these books, you may have included some Classroom Assessment Techniques in your teaching toolkit. Welcome, Tom.

Tom: Good to be here.

John: We’re very happy to talk to you. Today’s teas are:… Tom, are you drinking tea by any chance?

Tom: No, I was drinking black coffee.

Rebecca: It’s one of our most popular forms of tea [LAUGHTER] as regular listeners know. Today have some English tea time, John. How about yourself?

John: …and I have an Earl Grey tea today.

Tom: Excellent.

Rebecca: We’ve invited you here today, Tom, to discuss the third edition of Classroom Assessment Techniques, to be published shortly. These techniques described in earlier editions of this book have been widely adopted by faculty in all disciplines at all different types of institutions. And a lot has changed in higher education since 1993. But this book is still the go to reference on classroom assessment techniques, which is pretty impressive [LAUGHTER] for a book that was published 30 years ago. Can you talk a little bit about the origin of this book?

Tom: Well, I think the origin of the book goes back to the fact that when I was a young teacher, I recognized pretty early that many of the things that I was doing didn’t work and didn’t work for my students, and that I had to accept that things that had worked very well for me as a student weren’t working for many of them. So I began to experiment with asking questions in a variety of different ways and getting students to jot down answers anonymously. And through that, I really learned more about teaching and learning, I think, than from anything else that I did. And one of the things that I learned was not to assume. So, classroom assessment techniques are things that I’m sure almost every teacher has done. But what we did in the 1980s. K. Patricia Cross and I, was to really formalize those. And we did that in order to create a resource for teachers so that they could find simple tools that would help them ask students the kinds of questions that would help students learn better, and teachers teach better, as they went along. So it really began at that time, and it was kind of an accidental start, in many ways. We were doing it as part of a big project, and what was then the assessment movement, and it just kind of took off.

John: Most of our listeners are probably quite familiar with classroom assessment techniques and have seen them at lots of workshops and have probably, as Rebecca noted before, used them in the past. Could you define what a classroom assessment technique is, for those listeners who are new to the concept?

Tom: Well, in a formal way, a classroom assessment technique is a formative assessment technique. It’s a way of gathering data to use to improve and inform practice, rather than to make judgments. So we have too, most of us, to give students tests and exams and assign papers, and grade their work, evaluate their work, for grades and marks. Formative assessment techniques, on the other hand, are tools that we use to gather data along the way, before those moments in which we grade them, or mark them, so that we can help students get ready and improve and succeed when they’re going to be graded and marked and so that we can, at the same time, find out where they are and how they’re doing in order to help them improve along the way. So they’re really ways of diagnosing, in a way, students’ learning.

Rebecca: One of the most popular classroom assessment techniques described in your book is the one minute paper. There have been many studies of the effectiveness of this technique and it’s been widely adopted in many disciplines. Why was this technique, in particular, been used so extensively?

Tom: Because it’s simple and easy to use, probably, and it takes almost no time. But seriously, the one minute paper, the minute paper was invented, to the extent these things are invented by Charles Schwartz, in the 1970s, at UC Berkeley, and he used it as both a way to take attendance (in his case, he was asking students to jot their names down), but more importantly, to find out whether students were learning from his physics lectures, in his huge physics classes, what he hoped they were learning. And that idea spread through UC Berkeley really fast. And so soon, there were dozens and hundreds of people using these two questions. And the two questions, famously, were: what are the two or three most important things you learned from today’s lecture, in his case, and what questions do you still have the end of the lecture? So students would jot these down on those strips of paper, remember, there was no internet and no computers, really, in classrooms in the 1970s. And then he would collect them, and read through a sample of the 1000 or so that he had every lecture. And from that sample, he would gather ideas about what students understood and didn’t from the lecture, and also what questions were most common. And then he would begin his next class meeting, by going over some of those points that students had had trouble with, or not understood, or missed them completely. Sometimes it happened that a 1000 students didn’t get one point that he thought was critical, and also answering one or two of the most important questions. And what he found was, he could do all of that in five minutes at the end of class, and about five minutes at the beginning of class. And I think it’s that efficiency that has made it so popular.

John: My first exposure to this was a study that was published in the Journal of Economic Education way back around the time of your last edition. I did try to find it, but I couldn’t find the citation for it, but what struck me was that the instructor had used it in one section of the course, had not used it in another section, and found that there was a very substantial difference in learning outcomes between the two groups, which he attributed to the one minute paper. And it’s been incredibly commonly used. But one of the things that really struck me is that at the time when you came out with this book, including the one-minute paper, and so many other classroom assessment techniques, most instruction at the time was the chalk and talk type where people were at a blackboard, maybe a whiteboard, if they were a little bit more advanced in the technology, where most assessment was done with high-stakes assessments. So I think your book was a fairly important factor in helping people shift to more formative assessments. Why have these techniques become so widely adopted?

Tom: Well, I think that one of the reasons why people have found classroom assessment techniques and other formative assessment techniques, there are other kinds, so useful and so important, I think there are several reasons. One is that the student body in the United States and across the world has become ever more diverse in the last 30 or 40 years. And it turns out that no matter who we are, as teachers, most of our students are going to be pretty different in terms of their experiences and their interests and their skills than we are. And that level of difference has probably grown over the years, at least our awareness of it has grown. So I think most teachers now realize that it’s very important to check in on students and see how they’re doing, before we get to the midterm or before [LAUGHTER] we get to the end of the course, and to check in regularly and not to assume that students are understanding our brilliantly clear presentations or the material that we give them, or the readings, or the problem sets, and to really check early on to see if it’s making sense to them, and what kind of sense it’s making to them. So I think that there’s much more realization of the diversity of human learning, and of our students, than there was 40 years ago or 30 years ago. And I also think that as teaching has evolved, especially as it’s become online, people have recognized that we need data. We can’t just depend on students volunteering that data, they often don’t know that they don’t understand. And that that data, used well, can help us and can help them.

Rebecca: I think one of the things that you’re talking about in terms of assumptions is reminding me of some of the conversations that we had early on in the pandemic, where instructors we’re talking about not being able to see the sea of faces to know whether or not students are learning because maybe students aren’t nodding along or providing some visual feedback, but classroom assessment techniques are really important because they allow opportunities to gather that data, which is probably far more informative than the head nodding.

Tom: I think there are maybe a couple of other things that are important about this that people discover. One is that by stopping to ask students to reflect on their learning and respond, you give them time to think. And you prompt them to think about their learning. And from my point of view, forever, the most important thing about doing this has really been to give students tools to become more aware, we could say more metacognitively aware, of their own learning, and to better take responsibility and control or at least direction over their own learning. So what I found is I’ve had many, many, many very bright students who had never been taught any kinds of techniques to think, or to ask themselves questions, or to monitor their own attention, and having those sorts of tools, those sort of metacognitive tools, I think empowers students to make decisions about their own learning, and also to improve their own learning in those areas where it matters most to them, which may not be my course.

John: One of the things you mentioned was that the one minute paper was originally developed in a class with 1000 or so students. And one of the nice things about many of these techniques is that they can scale that way. Now, there was a time when the book came out when this was all done on paper. But now we have so many electronic tools that we can use to do this. Instructors could put up a QR code with a link to a Google form or something where students can give their feedback within a minute or so or we could use polling techniques, and so forth. And many polling systems have built in one-minute questions or some other classroom assessment technique as a default strategy. So this impact has been pretty substantial. And it’s really nice for instructors, not just for students, having that time for reflection, but also for instructors to get some feedback on what students have learned. Because if you just call on students, you may happen to get the only student who understands the material responding to a question or you might get the only student who hasn’t understood the material. But with these formative assessments, you can get a good feel for where students are. And they also get some feel for where they are in a way that they might not if they would just passively sitting there listening to a lecture. So these techniques have, I think, become increasingly more common from the time of introduction. So these techniques have held up pretty well. But you now have a new edition of this book, a third edition of this book. Could you describe some of the changes in the new addition?

Tom: Well, obviously, and you’ve touched on this, technology has changed. And that’s had an impact on teaching, and an impact on learning. So one of the things that I did, in going back to do this book after so many years, was to take a look at first what was out there, what had people published on this. And there are literally about 3000 published articles of various kinds and various quality, about classroom assessment techniques. There were about 300 that I thought were of sufficient kind of quality and rigor to be able to use. And in the chapter on the minute paper, I refer to 150 articles and studied them very carefully. There are many, many books that refer to the classroom assessment techniques. And I also read [LAUGHTER] all of those references. So I took a few years to work through this material and new things were coming in all the time. So one thing that differs is that we have an evidence base across many, many disciplines for what seems to work and what seems to work well. And also much useful information on how to use these with various technologies. So how best to use formative assessment with online, fully online technologies on totally asynchronous classes. And so that’s been really valuable. And those are experiences that I couldn’t have all of those experiences. So it’s terrific to draw in those from other teachers. The other thing is people have invented new techniques or developed new techniques over time. And so in the third edition 25%, or about 15 of the 55 techniques, are ones that didn’t exist in the second edition. And that’s about one quarter of all the techniques in the new book. So one of the reasons why we have different techniques in the 3rd edition is that some of the ones in the second edition weren’t much written about, weren’t much commented on, and they seem not to be much used as far as I could tell. So I’ve picked up the ones that were most used and most written about, and most referenced so that people in different disciplines can find those techniques. And I’ve also found another 15 techniques to kind of fill those gaps and extend it a bit.

Rebecca: Your book offers many, many assessment techniques. I’m curious what your favorite few are that you’d like to introduce instructors to?

Tom: Well, just like I can’t say which of my daughters are my favorite, I do have favorites. I’ll be honest. I have always used the minute paper ever since I learned about it all those years ago, and I learned about it really when I was at Berkeley, in the mid 1980s. And I learned about it, as everybody else there did, kind of by chance, and thought, “Wow, why didn’t I do this? Why didn’t I think of this?” So the minute paper, the muddiest point, which is a variant of the minute paper, which is developed by a really famous and eminent statistician at Harvard, named Frederick Mosteller. And Fred Mosteller heard Pat Cross talk about this, when we were both at Harvard, and being the kind of person that he was he immediately experimented with it. What he found was that it was more useful for him, in his courses, to just ask one question, and that question for him was “What was the muddiest point In today’s class? What was least clear to you?” Many other people have used that as well and written about that. That’s another favorite. A third one is one that I’ve used a lot, being a statistics teacher. And that’s predict, observe, explain, which is often called POE. And that’s one that I’ve found useful in many situations. And I’ve had colleagues who’ve used that even in literature courses, and other courses where they’ll say to students, “let’s stop at this point in the book or the film, and predict what’s going to happen next, write that down, then we’ll observe it, and then we’ll try to explain why that either met our predictions or didn’t meet our prediction.” And while it’s obvious how people do that in science, and it’s part of the scientific method, it can be used in all kinds of circumstances. And it’s one that I’ve used with teachers who I’ve worked with, about classroom assessment and about formative assessment. I’ve asked them to predict what they thought would happen, then observe it, and explain what did happen and whether or not that met their expectations. And that’s been a very useful tool for me, I think, and the teachers I’ve worked with. So those are three good ones. I’ll give you one more, if that’s helpful. And that’s something that we call the directed paraphrase. So three of those four were in the first and second versions of the book, predict, observe, explain is in the third edition, but directed paraphrase simply asks the questions that are sometimes called journalistic questions, who, what, when, where, why, who did what to whom, when, where, why, and how? And it asks students to answer those questions about some phenomenon that they’re observing, or to answer them about an essay that they want to write, answer them about any kind of question. So giving students a framework of questions that they can work through, to make sure that they really understood either a book that they’ve been reading or an experiment that they’ve been doing, has been really valuable to my students. And I’ve said to them, “Look, this is something that you can adapt and take with you anywhere to use to make sure that you understand what’s going on fully. And to find out which part of it maybe you really can’t answer, and need to do some more work on.

John: In terms of the predict, observe ,and explain thing. I took a MOOC that was put together by Dan Ariely on behavioral economics. And one of the things he talked about was how he would often present the results of an experiment where the results were not what most people would have expected. But then he’d asked people for their reactions only after he summarized the results of the experiment. And they all nodded and said, “Well, that’s exactly what they would have thought would have happened.” So he started revising his presentations, where he asked people to make a prediction, and then revealed the result. And it got people much more engaged. And the same type of thing is done by Eric Mazur with his polling technique, where he asked people to answer a question where the question is not obvious and where there’s going to be some disputes. And then once they’ve committed to an answer, they work with other people and he goes through it again, with a little bit of peer instruction. But the level of engagement when students have made some type of a prediction of what they think the answer is, and then they have an explanation of what the answer is either given by the instructor or by other students, the level of engagement and interest just seems to go up fairly dramatically. So I can see how that would be particularly effective.

Tom: Yes, and I’ve learned a lot from the work of Eric Mazur, as have many others. And I think that that’s a critical thing. Because one of the things that we need to help students with, and help ourselves with, is managing our inbuilt biases and that issue of when you hear something new, feeling that you’ve always known it, even though you haven’t, is a cognitive bias that’s common, I think, to all human beings, myself included. So doing something like that: predict, observe, explain routine gives us a chance to really check and see if we know the answer. And then to find the answer, or discover an answer. And then to realize, because we have a document, we have a record, we have data about what we knew in the first instance, to recognize that we’ve learned something and what we’ve learned and that we might have been misinformed or unclear in the beginning. And I’ve suggested to my colleagues, one of the reasons that students don’t value their education as much as sometimes we think they should, or value our course as much as they think they should, is because when they get to the end, they may think “I always knew that, that was obvious, I knew those things.” And so we can help students realize that there’s a start point. And then there are many points along the road to learning, and that they may have learned more than they thought they did in a given course or given program. And I think that’s important for us in terms of demonstrating the value of education. And I think it’s important for students in terms of understanding and developing some humility about how it is that we learn and how hard it is and how long it takes to learn anything that’s really valuable.

Rebecca: I think, in my experience, students really appreciate the opportunity to slow down and reflect on their learning, because they’re often not given that opportunity. And when you’re doing that throughout an entire semester, they really do get excited about how much they’re learning and they get energized by that and seeing that they can achieve things over and over and over again.

Tom: I agree, and what we found with most of the studies of classroom assessment techniques, the simple ones is: first, classroom assessment techniques can’t really be separated from everything else that’s going on in teaching, they’re just one little part of the whole teaching and learning spectrum. They, at best, seem to make about a 5- 8% difference in how much students learn, which is about half a grade. And I think that’s important, but I don’t want to overstate it. So they’re not a panacea and they’re not a silver bullet, but they’re something that can lift up, to some degree, students’ performance and their learning. And the last thing I’ll say is nothing works for everybody. And so there is no technique or no series of techniques that every single teacher or every single student that I’ve worked with found valuable. And so one of the points of having 55 techniques is to hope that there’s one or two or five in there that people will actually find that work for them.

Rebecca: So you just reminded us about there’s 55 techniques, how do you imagine an instructor using your book? Is it a read from the beginning to the end? Do you have it organized in ways that instructors can easily dip in and out?

Tom: I can’t imagine reading it from the beginning to the end, and I don’t think anyone ever has. That would surprise me, except maybe the proofreaders [LAUGHTER] and they weren’t worrying about what it meant. So it’s organized so that there are many ways, there are actually eight ways to find a technique that might be useful for a given situation or a given teacher. One of those ways is the simplest and that’s alphabetical. So if someone has mentioned a technique to you, you can easily find it that way. The others are a bit more complicated, but I think the most valuable for many people is an index of all the techniques that have been used by discipline and documented by discipline, and the examples by discipline in the book. And there are a couple of 100 examples in the book of how people use these in different disciplines, kind of from A to Z, from anthropology to zoology, and everything in between. So those are ways that many people use. There are also a couple of inventories in the book. One is something called the teaching goals inventory, which was in the first and second edition, and has been used in lots of ways by many people and written about and research has been done on the teaching goals inventory, and it is exactly what it sounds like, an inventory of what we think we’re trying to teach. For this book, I created what I think is kind of a more contemporary take on that, and that’s something that I call the course learning outcomes inventory, or CLOI, and it’s really flipping that and saying, okay, it’s important what we want to teach, but let’s look at it from the point of view of what do we want students to learn? And so the CLOI, the course learning outcomes inventory, the statements are statements of learning outcomes. There are 57, they’re divided up into kinds of learning outcomes, and people can use it as a self assessment. Those items both in the teaching goals inventory and the CLOI are all linked to classroom assessment techniques in an index. So if you pick number 47 goal or number 47 outcome, you can find techniques that are linked to that. Lastly, we’ve done indexing by Bloom’s Taxonomy, and by Fink’s taxonomy of significant learning outcomes, and also by the big solo taxonomy, which is one that’s much used in the Commonwealth world, in the British and Canadian and Australian and New Zealand systems. So there are eight ways to skin a CAT in this book, or to find a cat at least. And I hope those will be useful to folks. And I think different kinds of indexing work for different kinds of people.

John: You mentioned that when this book first came out these things were done on paper. But one of the things that’s grown quite a bit over the years, as you’ve mentioned, is online instruction. Could you tell us a little bit about how some of the techniques can be used in an asynchronous environment?

Tom: Well, I think there are huge advantages to the technology that we have now compared to having to do all this on paper, having to read all those papers after class, or all those index cards, in my case, after class and then waited until the next class, when I saw students again, the time lag was a real problem. So being able to respond almost immediately, or in relatively short time to students in asynchronous courses, to their questions, and to their confusions, and to also let them know that they’re learning some things really well… they really got that… and that they’ve come up with some ideas that I never would have thought of that are excellent. To do those things in real time, I think, is much more effective in terms of teaching and learning than waiting two or three days or a week. Also, as you mentioned, we have technologies for polling students and for analyzing their responses that are ever more sophisticated and powerful. And so those save teachers a lot of time and work. I was always going through things counting and deciding on the number of times people had mentioned things on paper. This can all be done and is all done for you if you want to do it. And that, again, allows teachers to focus on what really matters. And that is: what’s the meaning of this data? What are students telling me? …and to make decisions about when they’re going to respond and what they’re going to respond to, because you can’t respond to everything in data if you have more than two students. And most of us did, and do. So I think that asynchronous learning has huge advantages, it has some downsides, but it has huge advantages. And one of those probably is that it makes it more difficult to assume that by looking at students, you can tell that they’re learning or not, which I’ve always found an odd idea, but many of my colleagues at least used to seem to think that if students were nodding and looking at them, that they were just tracking with them and understood everything that they were saying. And I used to say to my colleagues, “I’ve known many smilers and nodders that probably were thinking of something else entirely, and classroom assessment proved that to me.

John: I’ve seen that so much. We’ve heard so many faculty during workshops over the years who say, “Well, I can tell whether they’re learning just by looking at the expressions on their faces.” And I’ve never found that. There are a lot of students who will be nodding and smiling. And then when you ask them a question, or they ask you a question, it’s really clear that they don’t have the basic idea. At an economics conference, I was presenting a paper and there was someone sitting in the first row taking furious notes while I was talking. So I went up to the person after and asked them what they were taking notes about, because I was curious. And they were actually preparing for a session later in the day. But they looked really engaged throughout the whole thing.

Tom: They were engaged, just not with your lecture. [LAUGHTER]

John: Yes.

Tom: So I think the converse is also true. And that is their students, and people, who don’t appear to be paying any attention at all, or who are frowning or who have had their head down, who are actually paying total attention and learning a lot. And that’s one of the things I learned as well, early in my career, was not to make assumptions based on those sort of superficial impressions. And it’s hard not to, but when you actually learn that some students who appear not to be engaged are engaged in their quiet way. And that’s a humbling lesson, or it was for me.

Rebecca: One of the things that I really like about your book, and also the way the collection of the CATS all in one place is the flexibility, the flexibility of navigate the book, the flexibility of options for the instructor, but also the flexibility of options for learners as well, in different contexts. So I just wanted to highlight that flexibility piece that we haven’t really talked about quite yet.

Tom: Well, that’s the hope. And if this is created as an ebook, and I expect it will be, then I think that will offer people even more flexibility and more sort of usability. It will be a more user friendly book. And again, there are huge advantages to the kinds of technologies that we have available now that we didn’t then. So I think the real key to this is one of the axioms in the book. And that is it’s really about adapting, not adopting. So there are many, many tools in there that people have adapted in a variety of ways, renamed, and that’s all fine. And that’s all exactly what should happen. You should find them, adapt them to their uses, and if they don’t like the names of the CATs, they can change the name.

John: Would this be a good subject for a faculty reading group where people from different disciplines could get together and talk about how either they’ve applied these things or how they might apply them in their classes?

Tom: Well, it has been from the beginning. And we began this whole process working with groups of teachers, initially in community colleges and state colleges, who were interested in assessing student learning. And, of course, the point being trying to improve student learning. And what we did basically was use the teaching goals inventory at that time, you could use the course learning outcomes inventory, with those teachers to get them to focus on what they really wanted to know. And in the book, I’ve written about, in one of the chapters, four different ways you can approach using classroom assessment techniques and figuring out which one to use. But everybody had ideas and theories and questions or problems that they wanted to try to solve. And so we started with that. And then we said, if that’s your question or problem or theory or outcome or goal, what might be a good tool to try to help you gather a reasonable amount, a manageable amount, of data from students that might help you learn, and might help them learn? And that’s really where we began, and those discussions and the collegiality that came from them, and the interactions that came from that, were always the most valuable part of that professional development. That when we ask people: “What did you take away from this?” The first thing that they said was not a bunch of techniques. The first thing that they said was the interactions with my colleagues, the relationships with my colleagues, learning from my colleagues, and then they talked about the actual content that they had learned and the things that they had done. So I think those interactions and those relationships that people can make, by having a group focused around this. This is an excuse to get together and talk about teaching and learning. I think from our experience, we found any excuse to get folks together to talk about teaching and learning, especially if it’s across disciplines, is always fruitful.

Tom: …particularly if there’s food. [LAUGHTER]

John: Yes. The third edition list you as author with Todd Zakrajsek, could you tell us how this collaboration came about and how it worked?

Tom: I think this was, in part, an insurance policy for the publishers. Given my advanced age, they wanted to make sure that they had someone involved in the project, who could take over if I just kicked the bucket. So, in the last six months of the book project, after I had finished writing the manuscript, Todd Zakrajsek joined with me and was really invaluable in finishing the process, in coming up with great ideas and suggesting great ideas, especially for the use of technology and AI with CATs, and in helping me manage the process with the publisher, which is much more complicated than it was 30 years ago. He also wrote the discussion questions for use by teachers and faculty developers, and I think those are going to be very helpful to folks.

John: We’re recording this in May, and the book is scheduled for release in June. We always end by asking, what’s next?

Tom: Well, I’ll answer that. But I’ll first say to teachers, what I always say, and that is, “Feel free to ask your librarian to order the book, so that you can use it, and your colleagues can use it.” Don’t tell Wiley and Jossey Bass that I said that, but I think that’s important. So what’s next for me is I’ve been working for a while with a group of people in, strangely enough, pharmacy education. I’m not a pharmacist, and I’m doing research with graduate students on what we call core concepts in developing what are the core concepts in pharmacotherapy? And that’s been very interesting for me. I’ve learned a lot from that. And so I’m ongoing with that. And we’re beginning to work on educational materials related to that. So that’s one of the things that I’m doing. And I’ve actually, I don’t want to commit to this too much, but I’m thinking about writing a book based on an article that I wrote that people have found useful, called “A Teacher’s Dozen.” And so I’ve updated that. I’ve been working on that actually for a few years, and something may come of that.

Rebecca: Well, we’ll be excited to find out what that is soon, maybe. But you haven’t committed, but maybe. [LAUGHTER]

John: Well, thank you. It’s been great talking to you. And my copy of the book is on preorder. And I am looking forward to receiving that.

Tom: So am I. [LAUGHTER]

Rebecca: Well, thank you so much for joining us. It was great to hear about the origin story of the book and how it’s evolved over time and I know a whole other generation of teachers will appreciate the new edition as well as teachers who have been practicing for a long time.

Tom: I hope it will be useful. Listen, it’s been a pleasure and a privilege to talk with you both. Thank you for giving me this opportunity

John: We should note that this is the first of two podcast episodes discussing this book. In next week’s podcast we’ll be joined by Todd Zakrajsek, who will be discussing his role in this work.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

345. New Era – New Urgency

Public confidence in the U.S. educational system has been declining while reports of student disengagement have been rising. In this episode, Deborah Pomeroy and F. Joseph Merlino join us to discuss the possibility of repurposing our educational system to better support the needs of our students and our society.

Deborah has over 50 years of education experience and is professor emeritus at Arcadia University. She has co-directed a Dewitt-Wallace grant, Students at the Center, for inner-city schools in Philadelphia and was actively engaged in the Bioko Biodiversity program in Equatorial Guinea. Joe Merlino has spent 39 years in education. He has been a principal or co-principal investigator and/or project director on numerous federal grants. He currently directs a seven-year USAID grant in Egypt where a team of US faculty are co-developing 180 new undergraduate STEM teacher-preparation courses for five large Egyptian universities.

Deborah and Joe are co-founders of The 21st Century Partnership for STEM Education where Joe has served as president since its founding in 2007. They are also the co-authors of New Era – New Urgency: The Case for Repurposing Education, which was recently released by Lexington Books.

Show Notes

Transcript

John: Public confidence in the U.S. educational system has been declining while reports of student disengagement have been rising. In this episode, we discuss the possibility of repurposing our educational system to better support the needs of our students and our society.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Deborah Pomeroy and F. Joseph Merlino. Deborah has over 50 years of education experience and is professor emeritus at Arcadia University. She has co-directed a Dewitt-Wallace grant, Students at the Center, for inner-city schools in Philadelphia and was actively engaged in the Bioko Biodiversity program in Equatorial Guinea. Joe Merlino has spent 39 years in education. He has been a principal or co-principal investigator and/or project director on numerous federal grants. He currently directs a seven-year USAID grant in Egypt where a team of US faculty are co-developing 180 new undergraduate STEM teacher-preparation courses for five large Egyptian universities.

Deborah and Joe are co-founders of The 21st Century Partnership for STEM Education where Joe has served as president since its founding in 2007. They are also the co-authors of New Era – New Urgency: The Case for Repurposing Education, which was recently released by Lexington Books.

Welcome, Deborah and Joe.

Deborah: It’s great to be here. Thank you so much.

Joe: Thank you.

Rebecca: Today’s teas are:… Deborah, are you drinking tea today?

Deborah: I am. I have a cup of chai right in front of me.

Rebecca: Perfect. How about you, Joe?

Joe: Yes, this is English breakfast tea.

Rebecca: Oh, yum.

John: And I have a black raspberry green tea today.

Rebecca: Nice. We got green tea in the office today. I have a raspberry Jasmine green tea today.

John: We’ve invited you here today to talk about your new book. Could you tell us a little bit about the origin of New Era – New Urgency: The case for repurposing education?

Deborah: Well, that’s an interesting story. We actually started the book over 14 years ago when we were working on the math-science partnership for Greater Philadelphia. And it was the culmination of work that both Joe and I had been doing, both independently and collaboratively, in education reform. And we were beginning to develop our ideas about education reform. And we’ll go into those in more detail later. But we started the book, and we actually got quite a bit of the book written. And then we were interrupted by a meeting with the Minister of Education from Egypt, who came to Philadelphia with a group of colleagues to study STEM schools in the US. And as a result of that meeting, we ended up working on this USAID project in Egypt, helping the ministry develop a series of model STEM schools. And when we got into the work there, which was basically 24/7, 365 days, and we were joking around that many times, it felt like we were drinking water from a firehose, it was so intensive. And we were never able to go back and finish our book. But as we were doing the work, we sort of looked at each other a number of times, sort of laughing and saying, “This is the last chapter of our book,” because in the last chapter, we hit envisioned what real transformation could be in education, and we were actually doing it and it was so exciting. And so we decided that we needed to recast the first part of the book, and then add this whole latter part of it, because we actually have a case where our ideas have been able to be put into place and have worked just amazingly well. And so the book is actually a total of 14 years in the making.

Rebecca: So I’m really curious about the word choice of repurposing because you could have probably picked many, many titles. Can you talk a little bit about why repurposing is your word choice in the title?

Joe: We felt, through our reform efforts that were difficult to implement, that the single most durable element that prevented reform from happening was the inability of teachers and students to articulate why they were teaching what they were teaching, as opposed to all the other possibilities of what you could teach per subject. So why are you teaching algebra? Why not statistics or probability? Why are you teaching it the way you’re teaching? So the idea of purpose, which is answering the why, was what we found to be lacking. That’s why we felt purpose was the central idea of the book.

Rebecca: I knew there was a key reason.

Deborah: When we started work in Egypt, the first questions that we asked them were: What are your aspirations for the future? And for the outcome of these schools? In other words, what do you dream these graduates could do and become, and when they were able to start to articulate that, and we’ll go into that in more detail later, then we built the entire curriculum and assessment system around that purpose. And so we found that that, in fact, was the linchpin for this transformative reform.

Joe: After our meeting in August of 2011. This is like a year after the Arab Spring in Egypt, that’s when the Egyptian delegation came to Philadelphia. Four months later, we found ourselves in Cairo, I did, and two other colleagues of mine. And so we’re sitting in the first school, which is outside of Giza, and it’s in the middle of nowhere. I mean, it’s 30 miles, it’s just dirt and dust in this building. And we’re sitting in the principal’s office, and looking around wondering what’s happening and in bursts the first deputy minister of education in Egypt, and he says, “Welcome to the dream.” [LAUGHTER] And we’re looking around, and we’re saying “there’s nothing here.” So we had the opportunity to rethink things, and for them to rethink things. And I’ll just say one other thing real quick, is that the purposes of education derives from your aspirations for your country or for your community, and that defines what your purpose is. And that’s what we did in Egypt.

John: And you begin the book by talking about how the purpose of education has evolved over time with a little bit of lag, because educational systems tend to adjust slowly. Could you describe how educational systems have responded to changing societal needs?

Joe: Okay, I’ll try to do this in 60 seconds, if I can, the 400 years. [LAUGHTER] For me, it was a tremendous learning experience. We have, in our time, the ability to have the internet, to have digitized libraries, and search engines. So you can go back and actually look at primary sources. So it became a real learning experience for me as I went through it. But we started with the idea that we’ve been in this business for a while now, from the early 80s to now, and we could see the development of the Nation at Risk archetype that has defined our era here. And we wondered about, are there other archetypes? Have there been other paradigms? Have there been other purposes of education other than, right now, preparing kids for the global economy, which was the idea of the Nation at Risk? So we started by looking at the earliest instances of purposes, and we found it was the religious schools in New England, the Calvinist idea of preparing kids to be a part of a religious community, salvation was the key. And then we looked at, “Well, why did that happen? Why was it salvation?” So we looked at the English history and why people came to the United States was to form a new Israel. That was the vision of the Puritans. And then we looked at how this may have changed over a single lifetime. We talk about in our lifetime, how much change there has been if you think about yourselves and your grandparents. So we looked at history through the lens of five lifetimes, laid end to end. We said, “Well, what happened during these lifetimes?” And you see a tremendous amount of change that happened, that’s what we described in each lifetime, but along with those changes have been associated changes in the purposes of education. So I’ll just give you a quick example. So from this idea of the Commonwealth of Massachusetts, from this idea of a religious community, came the idea that as more immigrants came in, there was different religious sects. And there was conflict between these sects, so much so that when William Penn came to Philadelphia in 1682, he brought the Quaker ethos, the idea of the Quaker schools which was based on religious tolerance, and peace. And so it became another purpose of education. And he founded the first Quaker school here in Philadelphia. So that’s an example. Another example is the industrial revolution. There couldn’t be industrial education if there wasn’t an industrial revolution. So that’s what we did.

Rebecca: It’s really done a good job of setting the stage of how change has occurred over time. Can you talk a little bit about how the current system aligns or does not align with our current societal needs?

Deborah: Right now, the paradigm that is pretty much driving education in the US is college and career success. And many schools that I’ve worked in, for instance, have that plus perhaps lifelong learning as part of their school mission and vision. But how does that define the curriculum as such, and there’s really a mismatch, and what would a curriculum look like if we’re really focused on lifelong learning? And while the college and career success is driven so much by standards and by test scores that really are quite empty when it comes to looking at what we need as a society going forward. And going forward, we need people who can collaborate together, we need people who are very well versed in assessing the truthfulness of information and resources, we need people who can listen, we need people who are critical thinkers in terms of being able to deconstruct problems and analyze them and look at their resources and so forth. We need people who know how to recover from a failure and actually see failure as a source of learning and from which positive things can come. Those qualities are not served by high test scores in math and reading and by a curriculum which is so focused on teaching a course to prepare students for the next course. Where does life come in? …students who understand citizenship, who understand the civics of our country, and the government, and our system of checks and balances within our nation and what it really means to be a responsible voter. And on top of that, our students are in a culture in which they are dealing with the effects of pandemics, they’re in a media culture, they’re in a culture of diversity, because our kids below the age of 18 are more brown than white, that demographic is increasing in the country and we’re going to become more and more diverse. We’re dealing already with major effects of climate change. On a more personal level, the kids are dealing with bullying and Infowars, drugs, gun violence, immigration, you name it, and where on earth does the current curriculum prepare students to deal with the culture that they are facing in their own lives and what we need as a country going forward?

John: And I think everything you’ve said are things that everyone would agree need to be addressed. And there have been lots of attempts to bring these things into the school system in terms of an emphasis on lifelong learning, critical thinking, an ability to interpret information, and yet we still see the same basic curriculum being used. Why have previous efforts at reform failed?

Joe: That was the basis of the book. That’s why we wrote the book, because we’ve been doing reform efforts for decades. But we’ve been operating within this bubble of this Nation at Risk archetype, and it’s not relevant to where students are and where our country is. So the curriculum has been set, and then there’s justifications for it, rather than starting at a blank slate and saying: What’s our aspirations? What do you want to do? and then working backwards and deconstructing the subjects to fit that purpose? Like, I’m a math teacher, or I used to be a math teacher. I’m a recovering math teacher, [LAUGHTER] but I also was a philosopher and a cognitive developmental psychology, etc. There are so many different subjects and areas of interest that could be taught, not just in math, but in everything. So why this? Why this curriculum? So when we redesigned the STEM teacher preparation program in Egypt, we had to design an entirely new program. So I went to Cal Poly who was one of our partners, and we had a design session with them. And so it was a group of people at Cal Poly. And I said to them, “we’re going to design a new teacher preparation program.” And I said, “Imagine a teacher preparation program with no courses. What would it look like?” And they were stunned at the question, but eventually, they came around and after three hours, they started to think about it. And they actually did come up with courses, but it was a whole different program than they would have done otherwise. And so I would challenge people to say, what if in high school, there was no college as there was in the first half of the 20th century, it was mostly just high school, there was no college. So what would high school look like? Why would it be there? So the impediment to reform when you get right down to it is the lack of a coordinated coherent philosophy of a school that can inform the curriculum choices that are made. And that this curriculum, that biology talks to chemistry and math talks to each, the humanities are included as a central idea of a school because one of the big problems of people is getting along with each other. So you need history, you need literature, you need to understand human behavior and what triggers you. This is absent, you’re not going to get that from precalculus. And I’m not saying that precalculus isn’t important. But Latin teachers would argue that Latin is important for different reasons. So this is why it comes down to what’s your vision for the kind of life you want to live and the kind of world you want to live in? And then let’s package that in a way that relates and is integral to a child’s life.

Deborah: So, in schools today, many teachers do try to include projects and problem solving with their kids. And we know that that’s very much a part of the pedagogy in many schools today. But the problem solving and the projects are, may be related to the subject matter, and may be of interest to the kids, and maybe not. But the way we did it in Egypt is the schools were designed around the premise that the graduates would be prepared to address the grand challenges of Egypt. And we have grand challenges like climate change, alternative energy, population growth, urban congestion, disease prevention, and so forth and so on. And so all of the courses are designed to provide students with the concepts and skills necessary to address these grand challenges. And so the projects that the students do are projects that are directly related to these grand challenges. And these challenges are things that the kids see every single day when they look out the windows or on their way to school. There are problems that are so pervasive that everyone in Egypt [LAUGHTER] looks around and says “Can’t we solve these problems.” And so what we’re doing is we told the kids, this is what we’re preparing you to do. And so when the kids undertake a project, or work on problems, they’re real, and they’re urgent, and the kids are empowered to think about the fact that they could become agents of change in their country. And that is so different from the kind of projects that we do in our schools today. And when I talk about all the challenges that we face in our society, and the kids are facing in their lives every day, those could be some of the kinds of challenges and problems that a curriculum is trying to address. And those are things that are very, very real and meaningful to the kids. And it sort of takes the kids back to when they were five and six years old and wanted to make lemonade stands to meet the needs of somebody in the community who needs money or something. That altruism, that wonderful altruism of smaller children that somehow gets taught out of them, or many of them, not all of them, certainly. But anyway, I’m trying to take what Joe is saying here and putting it into some very, very specific kinds of things that are so different between a school which is purpose driven, with a purpose which is meaningful, as opposed to a purpose which is really meaningless.

Rebecca: You described in Egypt kind of a context that was ripe for change, and ready to re-envision, repurpose, what would need to happen here to better align our schools for today’s needs in the US?

Joe: I’m glad you asked that. So we brought the process that we used back here to Philadelphia, and we convened a group of 120 people in eight design studios. We had a small grant to do this. And we asked them that question, we said, “What’s your vision for the greater Philadelphia area? What kind of society would you like to live in in the next 10 years?” And we spent time articulating that and then out of that came different purposes of education. It was, you might say, a mission statement… or more than a mission, it was a purpose statement that came out of that vision. And there was like eight elements of that purpose. I’ll give you two of those elements. One had to do with diversity, “how do you live in a diverse society?” And then the second one was, “What are the unifying elements for that diversity?” So if that is your top line theme, you might say then if you’re in biology, how would you exemplify that theme? Well, you talk about biodiversity. And then you talk about evolution as being the unifying principle of biology as one of those things. For chemistry, the diversity of the material world through the elements. And well, what’s the unifying principles to that? So you look at the natural world diversity, which is immense, and then you look at the human world of diversity. What unites us? We have tremendous diversities, not just around race and religion, but personalities and cognitive abilities. So it gets to then how do we live together among all of this kind of human and natural diversity? What unifies us? Kids want to know that. They have to deal with all of us, otherwise, you just get gangs. So that theme of unity and diversity can cut through all of the subjects, look at music, my God, and yet there’s a unifying principle. So those are the two elements, and so when you then sit a group of experts in a room that have diverse subjects, and you say, “Alright, here’s your theme: diversity, unity, you’re a biologist, tell us how would you structure the subjects along those two themes?” And we did the same. And out of that, then we create a curriculum. So it’s like a rug, warp and woof, the threads of a rug becomes an integrated curriculum from that, so that when a student comes into that school, they understand what the school is about, and why they’re learning what they’re learning.

John: How has that worked? How have the changes been received?

Joe: The people who participated were really amazed by the process, and they got a lot out of it. The next step in that is then developing the curriculum, and then implementing it. So we attempted to do that with the School District of Philadelphia. And they were not interested, even though we had this process. But why we were so thrilled and still are thrilled with our work in Egypt is that we talked directly to the Minister of Education, we were in meetings with him, we had his first deputy minister, who is still directing the project. So at the very highest level we had buy in. President El-Sisi visited one of our schools in Minya, Egypt and said, “My God, these students are amazing. I want 100 More of these schools.” And more recently, the minister and the head of the Parliament said we want to have this idea throughout all of our schools,17 million in Egypt, the elements of that idea. So what the biggest change has been in this process is, and the students will tell you, not me, the students will tell you is that they came in as individuals in a competitive environment. They came out of it, they had a personal transformation of seeing something greater than themselves, cooperating with each other, and thinking of it as “we” rather than “me.” And boy, wouldn’t that be great in this country if we had that spirit of let’s just work on this together?

Rebecca: I think your story really attests to the importance of some of the grassroots kind of components of reform as well as real support from leadership.

Joe: Exactly.

Rebecca: You also ventured into Bosnia Herzegovina. Can you talk a little bit about what that project was like there?

Joe: So a lot of this stuff happens through serendipity. We didn’t go out and apply for Egypt, they came to us, quite by accident. So the same thing happened in Bosnia Herzegovina which, as you know, it was a terrible situation back in 1991 with the breakup of Yugoslavia and tremendous genocide there. Save the Children had been working there on a USAID grant to work on a STEM school. This is in 2017. And it wasn’t working out too well. So they called on us, based upon our work in Egypt, and we went over there. And we did the same process with a group of university people and other officials of this design studio. We said, “What’s your aspiration for your country?” And their aspiration was: we want peace, and we have tremendous unemployment, and also people are leaving the country. So we want to be able to have peace, have people work together and not fight each other and be employed. So for them, they came up with the idea that the purpose of education is to be equipped to participate in a knowledge-based economy (KBE). I said fine. Well, what are they? So they listed 10 sectors of a knowledge-based economy? I said, “Fine, alright, so we have design studios, where we list these sectors and the elements of the sectors and then across this matrix was the subjects. And from that they developed a K to 13 curriculum that was integrated around these KBE sectors. So our role there was not to develop the curriculum, but to consult on the process by which they could do it themselves. And so we worked there for a couple of years. And we developed the curriculum app, an application, that was a relational database that had all of this curriculum in four different languages. So, it was interesting.

John: In your book, you advocate moving towards a more purpose-driven curriculum, but we’re living in a country where there’s a tremendous amount of political polarization and a lot of divisions, where there is a already a lot of pressure on schools to move in different directions tied to the local politics of people in that community. What can be done to move towards a consensus on the goals of the educational system in the US.

Deborah: There are a couple of principles that we need to deal with. One is that conversations have to happen across the sectors of community, whether we’re talking a local community, as a community of as large as a state, or national. One of the things of course, we don’t have a national education system as such here. The closest we get to a system is at the state level, but really, decisions are made much more at a local level. And we have tremendous diversity, even within the state of New York, for instance. I mean, you have one of the largest urban areas in the world, and then you have extremely rural areas, and with levels of affluence and poverty at the greatest extremes of the continuum. But conversations need to happen, but they need to be carefully facilitated, because you just don’t get in and say, “Well, what are we going to do?” You have to have very careful guidance and structure in this kind of design studio with everybody buying into the idea that things need to change. And we need to be able to really discuss freely and openly. And I think one of our beliefs is that when you have these discussions, I believe personally, and I’m a bit of an optimist in this respect, that we’ll find that many people, even from very diverse sectors, whether it’s politically or demographically or whatever, really have the same aspirations. I think the key is to identify the shared aspirations, and maybe where they aren’t shared, maybe there’s a way to forge unity between those. So I’m going to stop at this point, and then let Joe continue on with that.

Joe: I think that’s true. I think the key is to have a diversity of voices, and not just the loudest voices. And it needs to be, as Deborah said, very well facilitated in a process that doesn’t allow just the loudest voices to dominate. So that’s part of the process of developing that. But I think you also see that if you do a design studio in Oswego, for example, and then you also did it in Rochester, New York, and you did it in Hazleton, Pennsylvania, I would bet a bag of bagels that you’re going to find a lot more commonality [LAUGHTER] than you would usually think of it, because a lot of people just want to live, they want to have a family or they want to have success. There’s common human aspirations. But we live in a different time period than we did 1800s. So those aspirations have got to reflect the technology and the culture and the context of the times. And why that’s necessary is that you just can’t come in and impose a curriculum, you have to have buy in and acceptance… not to go back to Egypt, what we did was, we had to make sure that it was accepted. So if you have these aspirations, then you design the curriculum, you get affirmation from the curriculum, and everybody knows what the goal is. And once you decide you want to do it, and you have the authorization, if you will, from the decision makers at the policy level, financially, you can do it within a couple of years.

Rebecca: I love that you have great stories about success, and that it’s possible, because I think a lot of times it feels impossible when there’s a lot of division. One of the things that we’ve focused a lot on is the K-12 arena. What role does higher ed have in these conversations? Or what part of the conversation should we contribute to? Most of our audience are faculty and staff, administrators and things in higher ed institutions?

Deborah: I think higher ed has a huge role in several respects. Number one is if you change the way you’re teaching students so that they come out of their high schools with different sets of skills, and a different knowledge base and these kids go into their traditional university classes, there’s going to be a major disconnect. And again, I don’t want to beat the drum about Egypt, but very quickly a story that our students every semester, do a major semester-long, what we call a capstone project. And these are evaluated by outside evaluators. The very first time we did this, among the outside evaluators was a dean of engineering at the University of Cairo. And afterwards, she said, “Oh, my goodness, these kids are at the level of my master’s students or even above, and here they are just sophomores.” She said, “This is just amazing.” And I turned to her and I said, “These kids are going to be coming to your university in a couple of years. What are you going to do?” And she looks stunned. She was absolutely silent. She recognized the disconnect. And so that’s what we would call sort of a ground-up push for change, but there have to be other changes as well. Faculty need to be provided with a kind of support to do their own transformations in their courses and their pedagogy. And that has to be a huge change. But that won’t happen unless they see the need for change. And so that’s why we think actually starting in the high school is a great place to do that. But it also has to come from the leadership of the universities themselves from the top down, so we have all three working together. And of course, the first change has to be on teacher preparation. But changes have to happen in other courses as well, and those will be slower and probably take longer to implement. But the teacher preparation implications for this kind of curriculum, this kind of way of pedagogy and his way of assessment… we haven’t talked about that yet… are really quite significant.

Joe: You’re not going to be able to redesign schools, like we’ve been talking about, without the participation of universities for a number of reasons. One is that people look to universities for permission to think differently, they trust the expertise of professors. So they want to make sure that if they do this, it’s not going to hurt their child’s chances of college. So you need a partnership, a very strong partnership with the University to make that happen. And you need leadership from the President and Provost to give it the heft to make it happen. And so universities have been and would be indispensable to making this thing happen.

John: And many of the discussions at universities are along the same lines, in terms of the purposes that you’ve mentioned at the start of this recording session, it just hasn’t quite made it into the curriculum to the extent which might be needed to truly affect that type of change.

Joe: So in part of my research, I came across the report on general education that was done at Harvard in 1945, after World War Two, and it was about what you needed to live in a free society. And it was a very strong statement about the need to prepare students both at the college and secondary levels to live in a free society and what that meant, and how central the humanities were in that process, but also, that you cannot just teach math or the sciences for their own sake, that there has to be a moral arrow to it as well, because we saw in Germany in World War Two, how engineers and scientists were used in nefarious purposes. And I think universities are caught in this dilemma that students want to come to universities because they want to get a better job. But universities have a social mission too, and it’s more than just private gain that you’re doing with a student, you really want to transform that student into something more than who they are already and to enculturate them with a grander sense of obligation and duty. That’s really the value of higher education and the institutions, as I see it. And I see them as indispensable. So we wrote this book in a way to give a history and a sense of permission and a way to begin talking about these issues with some common language. That’s why we wrote it, because we realize universities have the same problem. Your Gen Ed courses, what do you have in those? But also institutionally, where are you going? What are you about?

Rebecca: I definitely agree with all the things that you’re saying. And I’m appreciative that you’ve written this book. One thing that Deborah had mentioned was assessment. So I wanted to see if we could pick up on that thread very quickly. So when we’re thinking about a purpose-driven mission and curriculum, how does that change the way we assess things? Assessment has been a big subject on the podcast more recently. And so I think it ties nicely into a lot of the things that we’ve been exploring.

Deborah: Well, I’m really glad you picked up on that, because that was absolutely critical. But what we have to do is we first have to decide what is success? And we can’t answer that question until we know what the purpose is. And I was just in a group of educators last night, and they were talking about what are the challenges of grade inflation. I said, “Well, what happens if we change the way we grade so that there can’t be such a thing as grade inflation?” And so what we’re talking about is a way to so radically change things like through the kind of evaluation I was mentioning, where students do projects and they have outside evaluators, is it complicated? Yes. Is it challenging? Yes. Is it worth it? Absolutely. Can you have grade inflation when you use rubrics and do blind evaluation and so forth? Yeah, yeah, you can get rid of those things. But the bottom line question is: how do you define success? And that has to be driven by the purpose which has been, as we have mentioned, deconstructed into what are the skills and concepts necessary to achieve your aspirations?

Joe: In the book we talked about the tyranny of math scores, that and English language arts has been the sine qua non of achievement and value in schools, and if you look at New York’s test scores, for example, for grades three to grade eight, math and science scores, the gap between low income and non-low income people stays the same from third grade all the way to eighth. And that’s predominantly the black and brown, so the gap remains the same. And if you just focus on math, you’re always going to be in a situation where you see these kids as deficient in some way. Whereas if you have the idea that there’s many different things that you can measure, other than math scores, that are of value, you open up the possibility of trying to find assets within students. And you also, through your assessment, you have to allow students to fail, and then to recover from that failure by learning something new, and not feeling that they are labeled… you know, it’s the growth mindset idea. So there has to be assessments that do that. And I’ll just say one last thing, you know, whenever someone learns something, particularly if it’s a misconception they have, and then they come to a correct understanding, there is a moment of vulnerability, where they say, “If I’m going to learn, I have to open myself up to different ways of thinking.” That’s an extremely vulnerable state. So if you have an assessment system that is judgmental, kids are not going to be learning, that’s why it stays the same in terms of the achievement gap.

John: I think there’s growing recognition of that in higher ed, which is why there’s so much discussion of alternative grading systems that encourage students to recognize that making mistakes is part of learning, and not penalizing them by what they come in not knowing but evaluating them based on their achievement by the end of the term.

Joe: Well, we’ve submitted a proposal with UC Berkeley on elementary assessment schemes using learning progressions as a framework so that if you’re traveling from Oswego to Albany, for example, let’s say Erie, Pennsylvania, you have a roadmap of where you’re going. And if you’re not quite in Erie, you’re somewhere in the middle. Well, that’s where you are. It’s not that you’re a failure to go to Erie. That’s where you are. So having this sense of measuring progress, but not evaluating it in terms of whether you’re failing or not, but you’re informing the student, “Okay, you have achieved this much. Here’s where you need to go next.”

Rebecca: Well, I’ve loved our conversation today. There’s lots to think about at an individual level and a social level, institutional level, policy level, there’s a lot of levels. But we always wrap up by asking what’s next?

Deborah: Well, our book has just come out. And so we are really interested in trying to start these conversations, just as you have given us the opportunity to do today and get people to start to think about the ideas that we have. We wrote it with a general audience in mind. But although it’s an academic book, definitely in that respect, it’s being marketed by our publisher as an academic book, but we really want to try to reach a more general audience and get these conversations started. So we want to spread the word, we want people to start thinking about some of these ideas, and maybe talking amongst each other in small book groups or something as: “What do you think the purpose should be?” …and start it both as a grassroots and at all levels, get these conversations going?

Joe: Yeah, for me, it’s not about telling someone what their purpose should be, or giving them our idea of what the purpose should be. I mean, I have my private idea, but more it’s the process by which it’s arrived at in a sense that it can be done. So we look at this as a sharing of our ideas and opportunities. We’ve been struggling with this for many, many years. So when we’ve had this kind of success through this process, that we want to share it. We don’t want to own it, we’re giving it away. That’s what the whole idea of the academy is about. So we’re hoping that through these interviews that we’re doing here, that people will find value in these and will read our book and have discussions about the process and about how things are in their own places. And that there hopefully will be a critical mass developed. So that’s what we’re doing right now.

John: Well, thank you for joining us. You’ve raised so many interesting questions that need to be addressed for education to be more effective. Thank you.

Deborah: Thank you.

Joe: Thank you. You’ve been tremendous hosts.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

344. Failing Our Future

The traditional grading system that we are all used to is of relatively recent historical origin. In this episode, Josh Eyler joins us to discuss research on problems associated with traditional grading systems and possible solutions at different scales and in different educational contexts.

Josh is the Director of Faculty Development, the Director of the ThinkForward Quality Enhancement Plan, and a faculty member in the Department of Teacher Education at the University of Mississippi. He is the author of How Humans Learn: The Science and Stories behind Effective Teaching and a forthcoming  book, Failing Our Future: How Grades Harm Students, and What We Can Do About It.

Show Notes

  • Eyler, Joshua R. (2024, forthcoming). Failing Our FutureL: How Grades Harm Students, and What We Can Do about It. John Hopkins University Press (pre-order link)
  • Eyler, J. R. (2018). How humans learn: The science and stories behind effective college teaching. West Virginia University Press.
  • Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J. K., Smith, L. F., … & Welsh, M. E. (2016). A century of grading research: Meaning and value in the most common educational measure. Review of educational research, 86(4), 803-848.

Transcript

John: The traditional grading system that we are all used to is of relatively recent historical origin. In this episode, we explore research on problems associated with traditional grading systems and possible solutions at different scales and in different educational contexts.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Josh Eyler. Josh is the Director of Faculty Development, the Director of the ThinkForward Quality Enhancement Plan, and a faculty member in the Department of Teacher Education at the University of Mississippi. He is the author of How Humans Learn: The Science and Stories behind Effective Teaching and a forthcoming book, Failing Our Future: How Grades Harm Students, and What We Can Do About It. Welcome back, Josh.

Josh: Thank you. Thank you. Great to be here.

John: We’re happy to talk to you. And it’s been a while. Today’s teas are:… Josh, are you drinking tea with us today?

Josh: Well, you know how I roll, you guys, I don’t have tea, but I have some lovely water. [LAUGHTER]

Rebecca: So tasty. I have a Scottish afternoon today.

John: And I’m trying to cut back on caffeine, so I just have a peppermint tea today.

Rebecca: Well, if you wanted to cut back on caffeine, you could join Josh with water. [LAUGHTER]

Josh: That’s true. [LAUGHTER] Come join the club, John. [LAUGHTER]

John: I suppose that probably would not be an unhealthy thing to do. [LAUGHTER] And I have been drinking a lot more water recently.

Rebecca: So, we invited you here today to discuss Failing our Future. Could you tell us a little bit about how this book project came about?

Josh: Sure. Yes. So it has a couple of origin stories, actually. One is that when I was writing How Humans Learn a few years ago, I was working on the final chapter on failure. And I kept coming across research on grades and grading and the obstacles that they set up to learning. And certainly if you’re thinking about failure as a tool for learning, you immediately are confronted with the fact that in systems that prioritize grades, they push back on the natural cycle of learning, where we try something, we fail, we get feedback, we try it again. Grades are set up in the opposite direction, they arrest that process before it can really play out. And so that is what piqued my interest. And I don’t know that I could say honestly that I thought I was going to write a book about grades at that point. But I knew that it was an important element of what I was discussing. So I’d say a few years after that, I think what we were seeing was a lot of discussion about grades, a lot more experimentation with it. That led me to want to write a book for a broader audience, one that certainly included educators, but one that cut across the K-12, higher ed divide one that was also for parents and policymakers and students, one that really took a bird’s eye view of the larger conversation about grades, trying to pull all of what we know together so that we could move forward as a community to try and enact change. So that’s what got us here. And the process was certainly longer than I expected due to the pandemic, but it was also just a very illuminating process. And I heard some heartbreaking stories as I did interviews for this. But overall, I’m really proud of the work.

John: And you mentioned addressing this through the entire educational spectrum. And I think it’s important to address it through the whole system. Because by the time we see students in college, they’ve already been indoctrinated into a system of grades, and there’s a lot of resistance to change, and also a lot of damage, perhaps, may have been done by the use of grading systems early. Could you talk a little bit about how we ended up with this system of grading, which has been so much a part of the educational culture from kindergarten on up?

Josh: Sure, yeah. And you’re right about the fact that students are conditioned for at least 12 years before they get to us to think about grades as being the most important thing about education. So how did we end up here, we could have a podcast episode that would last five hours if we really wanted to dig into that. But I’ll focus really on the more recent history. And that is that the A through F letter grade scale that we have is really about 125 years old. Our first records of a full letter grading scale come from Mount Holyoke in the 1890s. So they were the first to implement it. Back then it was A through E. They later dropped the E because they were afraid that people would think that they were excellent when they were not, you know, God forbid that we allow people to think that they’re excellent. And so they dropped the E and added a letter where there could be no mistake about what it meant, and that was an F, for failure. So that’s when it began. But that system, the letter grade system, was not formalized, or standardized, really until the 1940s. That is when you see a majority, in fact, of school districts and colleges and universities adopt that system. And what I think is most important about that, actually two things. One is that none of this has ever been inevitable, that the 1940s is less than 100 years ago, so this is not something that was cast in stone at the minute the American educational system began; it has changed and can be changed. The second thing, though, is that all of these schools, districts, all of these colleges adopted that system, not because they thought it was the best way to document student learning, but simply because it was the easiest way to communicate across institutions. They felt that if they had a common system that they could communicate more effectively between themselves. So it was not about student learning. It was not about student progress or growth or development. It was simply about communicating. That’s how we got here. And now we’re trying to figure out where do we go next? How do we undo some of the damage that has happened since that time?

Rebecca: Speaking of damage, do you want to underscore what some of that damage is? [LAUGHTER]

Josh: Sure, and I know that there’s room for reasonable debate about the terms that we use here. But I honestly really do think we can use the framing of damage and harm when it comes to grades. And in the book, I have several categories. The first and most obvious category for that damage has to do with learning, that grades set up obstacles to learning. They inhibit curiosity, they inhibit creativity, they inhibit risk taking, they affect intrinsic motivation. This is painfully obvious, but grades are classic extrinsic motivators, which means that they are very good for getting people to do things that they would not otherwise want to do. Extrinsic motivators operate effectively, if your goal is compliance, you want to get people in seats, you want to get people to turn things in on time, you want to get people to say things in class, grades work for that. But if what you’re interested in is learning and quality, that’s what you need intrinsic motivation for. The grades can get people into learning spaces, but they in no way ever guarantee that a student will learn even though they are actually in the environment. So that’s really important. Environments that prioritize grades also incentivize cheating, and they just stand in the way of a lot of different things. But that’s just the academic part of it. Obviously, those are important, but to me, they’re less important than some of the effects of grades on students’ lives and wellbeing. Grades magnify and mirror inequities that have always been a part of the American educational system. For example, students who attend poorly resourced high schools have fewer educational opportunities, fewer textbooks, more teacher turnover, all the things that are tied to less-resourced schools. And what that means often when they get to college, they have experienced what we call opportunity gaps, which show up in their grades, especially in Gen Ed courses. So those grades that you see in the first couple of years of a student’s college experience, one who has experienced opportunity gaps, those are reflections of the past, they’re not indications of the student’s potential. They are indicators of systemic inequity. So that’s just one of many examples that we could rifle through. Grades are surveillance technologies. They’re often used for punishment, especially in high school, but kind of across the board. So that’s another category. inequities. And then the final one, the one that we’re getting more and more research on now, maybe than we’ve had in the past, and one that I personally care a lot about is that we have evidence now that grades are a contributing factor to the mental health crisis in teenagers and young adults, the academic stress caused by grades. We have a number of research studies pointing us toward the significant contribution of grades to that mental health crisis. So lots of areas here, I think, beyond the classroom. And that’s really our central idea of this book that grades have long afterlives, well beyond a semester, well beyond the classroom, that affect people’s lives, sometimes just in the short term, but often in the longer term as well.

Rebecca: Josh, you went from going from grades are no fun to grades are a super downer. [LAUGHTER]

Josh: Yeah, well, that’s the message. It’s a happy book, Rebecca. [LAUGHTER] It’s not going to win any awards for a happy smiley story, that’s for sure. [LAUGHTER]

John: One concern that some faculty have about moving to alternative grading systems is that, while intrinsic motivation is something we’d all really love to see in our students, there may be some tasks in our disciplines that students have to master which may not be as intrinsically rewarding as others, but they might still be foundational for students to be successful. If we move away from grading systems, what type of motivation can be provided to help students master those tasks that may seem tedious and may not be quite as intrinsically rewarding as other areas that students just find much more interesting?

Josh: Well, that’s a great question. There are a couple of things I’d like to address about this. So first of all, it’s not that we’re talking about either having traditional grades or having no grades, and nothing in between. What a lot of people are doing in the area of alternative grading practices is experimenting with a whole range of models. And so particularly for the types of courses and disciplines that you’re talking about in that question, John, I have found that many faculty gravitate toward a model called standards-based grading, where you can identify the skills, the content knowledge, the disciplinary norms and necessities that are important for students to develop. And the grade for the course is based, not on individual performance on exams, but how many standards they meet, over the course of the semester. And as a feature of that particular model, often students are given multiple attempts and multiple ways to meet the standards. So that’s part of releasing the pressure valve a little bit for students that gives them room to grow and honors the fact that individuals learn at different paces. So it doesn’t matter when in the semester that they hit the number of standards in order to get their A or B, because the process is a fundamental part of that grading model. So I think that’s really important. Another thing that often comes up in these conversations, though, is the fact that we do in fact, have gradeless colleges in America. The one I feature in the book is Evergreen State College in Washington. And they don’t have any grades at all. They have fully narrative transcripts. They give only feedback over the course of the semester. And they teach all the disciplines. So they have found ways to heighten the intrinsic motivation, and to use feedback to really help students navigate the path toward the goals that they have for themselves and for the class. So there are ways to do this. You don’t need grades to keep people in the seats or to have them do things that might draw on less intrinsic motivation. I think there are ways to structure our learning environments that allow us to do similar work without the pressure of traditional grades. And I think that’s the ultimate point here. I think what a lot of folks in this area who are interested in grading reform are trying to promote is not necessarily just ditching grades outright, although we could talk about that if you want, but reorienting students relationship with grades, the messages that grades send to students, so that it’s not necessarily an evaluation, but a tool helping them develop their skills and to push their learning forward.

Rebecca: It’s always been interesting to me that the A through F system was meant to provide some standardized ways of communicating between institutions yet, if you look at any single institutions’ body of syllabi, you can see that the grades mean entirely different things [LAUGHTER] depending on a class, like some might be about attendance, some might be about success on a test, some might be about achieving learning objectives. So it seems like if it was meant to communicate anything [LAUGHTER] across institutions, it certainly isn’t meeting that objective. I don’t know what my question is. [LAUGHTER]

Josh: I’m so glad this up, though.

Rebecca: I had a question, but I think I lost it. [LAUGHTER]

Josh: No, I’ll answer it. There’s a question in there, Rebecca.

Rebecca: Thanks for helping.

Josh: What you’re talking about there is what I call the measurement fallacy. And so there has been more and more work over the last few years, including one that I absolutely love, called A Century of Grading Research, where a team of experts dives into a massive amount of material. And the conclusion that they come up with is that a grade is nothing more than a subjective indicator by an individual instructor on a student’s progress toward that individual’s goals for a particular course. So I set the goals for my course, a grade that I would give is nothing more than an indicator of how much progress a student has made on my goals for my course. They call them learning intentions in this paper. It is not in any way, a kind of universal certification of learning in a particular course or discipline. Nor does it mean that the student who gets a certain grade in my course would get the same grade from my colleague down the hall teaching the same course. It is a truly subjective indicator of progress. And because of that, we cannot really say that grades measure what we have been told that they measure and what our educational systems have assumed that they measure, which is learning. They’re not universal measurements of learning, they’re subjective indicators of progress in the course.

Rebecca: And yet they’re used to make all kinds of decisions about students.

Josh: They are, and you’re absolutely right.

Rebecca: It’s very interesting. [LAUGHTER]

Josh: So I think there has been a lot of ill placed weight on grades without a subsequent amount of investigation into what grades have meant, and continue to mean. This is also what fuels the furor over supposed grade inflation. And there are lots of other tendrils of problems, when you begin to put so much emphasis on grades to do so much work in our educational system.

John: You mentioned Evergreen State College, but another college that had gone in that direction, very successfully was New College, and I teach in a program at Duke in the summer, and I had two TAs from New College, and they were two of the brightest people I had ever worked with, and their mastery of the subject matter was very much equivalent to that of other students I had who came from institutions with very traditional grading systems. So they certainly didn’t seem to be damaged by that. And yet, there seemed to have been a bit of a hostile takeover of New College. [LAUGHTER]

Josh: Yes, there has been but there is not evidence yet that the hostile takeover has affected that gradeless model. There was some discussion that I read in transcripts of board meetings about that grading system, but it has not changed yet. And I’m glad you brought up New College, John, because, and they have this on their website, as recently as last year, they were able to boast that they were producing students who went on to get the highest percentage of STEM PhDs in the country, relative to their size as an institution, which says a lot. If you are a gradeless institution, and your students are succeeding at that rate in STEM PhDs, it means that they’re learning not just content, but also the skills that are necessary to do that level of advanced work. So I think that that says a lot for what is possible, even within systems that do not have anything close to a traditional grading system.

John: And I should note that one of those students did go on to a PhD in economics. And the other student went on and picked up a master’s degree in data analytics. And they’ve succeeded very nicely, despite the absence of grades [LAUGHTER] in their college career. [LAUGHTER]

Josh: Imagine that.

John: When we began this discussion, one of the things you noted was that under traditional grading systems, the grades at the end of courses often reflect differences in the opportunities that students face before they enter our classes. Can you talk a little bit more about that?

Josh: Absolutely. So you have the reinforcing of those gaps, you have extensions of those gaps. So if you think about what happens to a student who comes into a college course with those opportunity gaps, and then is in a system, for example, where grades are curved, so not only then are they trying to bridge over those equity gaps, but their grades are really reflective of those gaps. But now they are also competing in a curved grade system with their other fellow students, many of whom have had more practice at the kinds of exams and the kinds of questions and the kinds of teaching strategies that they’ll see in those courses. So it just starts to pile on top of each other. And I think that this is something that we just don’t talk enough about, it’s just assumed, and many of these students are labeled unprepared or not ready for college, which places all the burden on them rather than this more systemic problem that I think we really need to face. Because we don’t talk enough about it, we assume that grades have to be a part of the landscape of education. And because of that, we, and I say we as the community in education, we’ve just been willing to tolerate all of the issues that grades have brought with them without really pulling back the curtain and looking at what the issues really are.

John: And we lose so many students in the first couple of years of college and many of the students we lose are those who come from low-income school districts, whose parents had less education, and who just come in with less preparation. And that’s not a very equitable way of providing an education.

Josh: Right, absolutely. And so this shows up on all kinds of metrics that colleges care about: retention, progress, graduation, all of which, by the way, are tied to grades and GPAs. That grades are the gatekeepers for keeping those students enrolled and moving through to graduation. So yes, that’s a major problem to address with respect to those students who are coming to us with fewer opportunities and from school districts with fewer resources. I think we need to really investigate the traditional ways of being In higher education, really look closely at them, reimagine them in order to create systems that have equity at the center and where we are actually, I think, invested in student success and helping those students from those different backgrounds to succeed.

Rebecca: I think the other thing to underscore related to equity issues is not only does an institution maybe have policies about grades and decisions are made at an institutional level, but at the federal level grades are used for financial aid. And so if you don’t hit a certain grade point average, and if you don’t have a certain number of classes that you complete successfully, so you’re not withdrawing from them, or having incompletes, etc, then you might not be eligible, you could lose financial aid. So whether or not a student has demonstrated growth in any kind of way doesn’t matter. It’s down to this idea of grades. And depending on the classes they’re enrolled in and the different things that grades mean, we’re essentially not allowing students to proceed because they can’t afford to.

Josh: That’s absolutely right. The more you really take the magnifying glass to this, the more you see how implanted grades are in all of these decisions, and all these key points in a student’s academic career, bringing in financial aid, I think, is a wonderful example of their gatekeeping function. And the idea that you’re raising there, that it doesn’t even give students a chance to try and bridge those equity gaps. It doesn’t honor their growth or development at all, it says you got less than a C average in your first year, therefore, we are not giving you any more money to move through the university, and so many leave, many dropout. And I think that there is a better way to do things.

John: And many of those students who leave end up with a fairly significant burden of student debt that they still have to pay, further increasing the inequity. What types of alternative grading systems do you discuss in your book, you mentioned standards based grading, but what are some of the other areas that you encourage faculty to consider?

Josh: Well, the book is divided into two parts. And the first half is all the problems and the second half is possible solutions. And so there’s a chapter for parents on what you can do in the home to help your children reorient their own approach to grades. And then the last chapter is about systemic change. But the one in between that is focused on what teachers in K-12 and faculty in higher ed are doing in their own individual classrooms to help with this. And so, there’s a whole range. There are the models that are related to standards-based grading, I call them the cousins: they’re standards-based grading, specifications grading, competency-based, mastery-based, proficiency-based, they’re all not very different from each other, they just have slightly different names, and maybe a few modestly different features to them. So that’s one bucket. Portfolio grading is one that I recommend for folks who just want to dip their toe in the world of alternative grading, because a portfolio model, you’re giving a lot of feedback throughout the semester, a lot of opportunity to revise and redo assignments. But ultimately, the portfolio that is turned in at the end of the semester gets a fairly traditional grade associated with it, it’s just that that grade honors the whole arc of the work for the semester. So that’s another one. Contract grading, and its various branches that we have recently seen. So classic contract model was developed in the middle of the 20th century. And it looks something like this: here’s a list of things to do for the course, if you do 15 of them at a satisfactory level… I’m just making up that number…if you do 15 at a satisfactory level, you get an A for the course, you do 12 you get a B, etc. That’s a contract model. More recently, there’s been a version of it called labor-based contract grading,…very prominent in writing studies… that tweaks it a little bit to emphasize more the amount of work, the amount of writing that students do over the course of the semester, and that is tied to fulfilling the particular contract. And then if we’re looking at a spectrum of grading models, with portfolio grading being one of the more conservative of the alternatives, on the other end of the spectrum you have what some people call ungrading and other people call collaborative grading where there are no grades throughout the semester, only feedback, lots of student self assessment, self evaluation, and then at the end of the semester, the students proposed their grade and the final grade is determined through a conference between the instructor and the student about that proposed grade. So there are lots of things that people are trying. I tried to send two messages about this, one is that my goal is hopefully to help faculty find models that will work best for them and their students, that it’s not a one-size-fits-all model. The same thing that works for me may not work for my colleague in the next classroom over. So that’s important. The other thing is that you do not need absolute fidelity to any one of these models. It’s not like Moses came down from the mountain, with tablets that you had to follow to the letter, lots of people are mixing and matching and taking elements from a host of different systems that they find works for them. So I’ve yet to find someone marching down the checklist for engraving or for contract grading, everyone’s kind of putting their own spin on it, and that’s good. Well, it’s actually not only good, it’s important since our contexts vary so significantly across higher ed.

Rebecca: As a lot of faculty start considering these different alternative models, collectively we can start to initiate culture change, kind of with a grassroots model. But systemic change requires policy change and other things as well. Can you talk a little bit about what role we might have in pushing forward some of those agendas as well?

Josh: Like I mentioned before, a major reason for writing this book was because my observation was that a lot of people are talking about grading reform, but they’re all coming at it, understandably, from their own perspective, the faculty perspective, the administrative perspective, the policymaker perspective, and the parent’s perspective. And I think that if we’re thinking seriously about systemic change, we need to have everyone on the same page collaborating to move change forward. All that is to say that I think about this question quite a lot. There’s certainly examples that we have of recent systemic change, some of the University of California schools have changed their grading systems in significant ways. Western Oregon just dropped the lowest grades on the scale, they’re no longer available to give. I just met someone a couple of weeks ago from Bryant University, they’re moving to collaborative grading at scale for their first-year seminar courses. So it is happening, what it requires is a network of people across an institution, all of whom believe that this is not only the right thing, but they understand how to use policy to make the reform happen, but also how to tell the story in a way that all the different constituencies can see themselves in it, and can agree that this is the direction that they need to head. So the networking is important, understanding how policy shapes grading habits and processes at a university. If you’re part of a public system, which I know you and I both are, you need to understand how the system policy and the state policy plays into this as well. So that requires having that network of different people at different positions in a university to really move that work forward. But what I really want folks to know is that you’re not alone in this, that there are lots of people thinking about this, and lots of people who are doing this work, sometimes under the radar, sometimes above the radar at institutions, and that one person taking a step is a step toward change. But multiple people taking the same step together begins to create a movement toward change. And so I want people to know that. I also want people to know that there are blueprints out there for how to do this work, that there are people who study grading reform, there are models out there for successful institutional reform. Given my work for the book, I think we have a lot to learn from K-12 school districts. There are so many of those each year that are transitioning to standards-based grading across the district. Last year, for example, the entire city of Santa Fe, New Mexico, all their school districts moved to standards-based grading. So this is happening. And the more we learn about process, the more we draw on our colleagues’ network and expertise in these areas, the more we work kind of as a collective in higher ed to push this forward, the more change you’re going to see.

John: We give a lot of workshops on campus and some of them are focused on alternative grading systems. We haven’t seen quite as much movement with that among faculty as I thought we perhaps might, because during the pandemic, a lot of people were open to trying some very different things. But there’s been some resistance. We’ve had a number of people trying it, but it hasn’t caught on quite as much as I had expected it to. Why are so many faculty resistant to moving away from traditional grading systems?

Josh: Well,I think this varies from place to place. But I will say that I think that a major thing that we all face as faculty is the pressures of time. Everyone wants to do their best for their students, I really do believe that, but when you’re faced with such a limited amount of time in which you can change elements of your course, you’re probably going to gravitate immediately toward the kinds of content that you’re teaching in the course in the assignments, the course design pieces of it, or you’re going to think about active learning strategy that John and Rebecca talked about a few months ago, rather than addressing the thing that is at the very foundation of our educational systems in America. That’s not going to be the place that you go first, [LAUGHTER] kind of an upheaval of everything that you have known about education. So I think some of it is just, in a system where we all have very limited time, and very few resources, where do you put your efforts, and I think that is driving some of that. I also think we’re kind of seeing this interesting bounce back that followed that period of innovation at the beginning of the pandemic. I think what you see is almost a defensive response to that period of innovation, that what we really need to do to bring students back and re-engage them in the coursework is to try and somehow recapture what our image was a traditional education pre-2020, that we need to move backward toward that rather than capitalize and extend the innovations that we were doing in the early stages of the pandemic. I don’t want to paint with too broad a brush on that, but I have seen some of that elasticity, that rubberband effect, in talking with folks across the country about some of these issues. So I think those two things are at the forefront of why some folks might resist that. And none of those really has to do yet with really trying to unpack people’s own philosophies of education, or how their grades and their grading models tie into their values that they have with respect to education. So that’s even kind of a deeper level that you can’t even get to until you address some of these other concerns.

Rebecca: And when we’re crunched on time, things that we’ve done before are easier to implement. And it’s also hard to find time to reflect on our practices and to do some of the things that you’re talking about, in thinking about how our values align with what we’re doing. We’re really good as institutions of talking about equity, but we don’t always fully analyze how our practices impact actual equity.

John: And we were all the products of a graded system. And we were somehow successful in that, or we wouldn’t be in these positions. And so I think there may be a little bit of psychological resistance to changing something that has been so fundamental to our own experience.

Josh: Yes. I think a huge capital Y-E-S [LAUGHTER] to both of what you’re saying, that Rebecca, your point, we are not very good at really interrogating how our practices may or may not be aiding our equity goals or advancing our equity goals. And John, yes, we have succeeded in graded systems. And I do think the psychological is a part of it. If we were able to do it, why shouldn’t other people be able to manage this? I think that is what some people think. And more than that, I often see a kind of defensive fallback. Part of this is natural, in that we love our disciplines and devoted our lives to our disciplines, but a fallback toward if I get rid of grades or if I change my grading system, what does that mean for the standards of my discipline, for how people will engage with my discipline? And so I think that’s a psychological element of this as well. And it comes out in arguments about rigor and standards, but what it’s really about is the individual psychology, I think, of the person making the argument.

John: Are there other topics from your book that you’d like to discuss that we haven’t?

Josh: I think the reform efforts that I have seen succeed, and those that have failed, always have hinged on communication, at every level. Why are we doing this? What is the purpose of it? How do we bring people into the process very early on at all levels. At the K-12. level, that means parents, teachers and administrators working together. I think at higher ed, it means utilizing the expertise of faculty right out of the gate, allowing them to shape the narrative and the conversation and the direction and draw on administrative and staff resources to help enact that vision, rather than having a top-down mandate that you had to change your grading. That’s never gonna work in higher ed. But I will say that the communication is what drives all of that work. And it is the hinge that either allows one of these efforts to succeed or fail.

Rebecca: Well, we have a lot to think about, but what’s next, Josh?

Josh: So, what’s next? Well, the book comes out at the end of August and so I’m really excited to launch that and to finally have it in people’s hands and have lots of conversations about what’s in there, hopefully, So, that’s what’s immediately next. I am thinking about the next project that has something to do with the students who are coming to college now are the students who their entire K-12 experience has been shaped by the Common Core, and the obsession with standardization in America, and so what does that mean for higher ed? And what does that mean for those students and their learning? So, I’m in the very early stages of trying to think about that a little bit, but that might be what the next project is.

Rebecca: You always have great projects, Josh, and get us thinking about really important topics. So thanks for your work. I’m looking forward to sharing your new book.

Josh: Well, thank you both very much. It’s always a pleasure to be on this podcast, one of my favorites, and I’d love talking to you both about all these issues, so hanks very much.

John: It’s great talking to you again, and I’m looking forward to receiving my copy of this book in the summer. I ordered it as soon as I saw the notification that pre-orders were available.

Josh: I appreciate that, John, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

343. Writers’ Groups

Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, James Lang, Sarah Rose Cavanagh, and Mike Land join us to discuss their highly productive long-term writing group.

Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College.

Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego.

Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University.

Show Notes

  • Cavanagh, S. R. (2016). The spark of learning: Energizing the college classroom with the science of emotion. West Virginia University Press.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed. John Wiley & Sons.
  • Jensen, J. (2020). Write no matter what: Advice for academics. University of Chicago Press.
  • Scrivener
  • A General Education – Jim Lang’s substack account
  • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge. Beacon Press.
  • Reading groups at Oswego (in the last three we were joined by Jessamyn Neuhaus and colleagues from SUNY Plattsburgh):
    • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning.
    • Lang, J. M. (2020). Distracted: Why students can’t focus and what you can do about it. Hachette UK.
    • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed.
    • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge.

Transcript

John: Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, we explore a highly productive long-term writing group.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are James Lang, Sarah Rose Cavanagh, and Mike Land. Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College. Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego. Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University. Welcome back, Jim and Sarah and welcome, Mike.

Sarah: Thank you.

Jim: Thank you.

Mike: Thank you.

Rebecca: Today’s teas are: …Sarah, are you drinking tea today?

Sarah: Never. You know me. I’m drinking seltzer. It’s too late in the day for coffee. [LAUGHTER]

Rebecca: How about you, Jim?

Jim: I had several cups of tea already today, it’s orange pekoe? And then my last one was English Breakfast actually at about one o’clock.

Rebecca: Nice. How about you Mike?

Mike: I am sampling some Bengal spice which the box tells me is brimming cinnamon, ginger, cardamom, and cloves and it’s an adventurous bland.

Rebecca: Does it taste adventurous?

Mike: It does. Everything about this is an adventure to me. [LAUGHTER]

John: And I have a blueberry green tea today.

Rebecca: And I have not my favorite tea. [LAUGHTER]

John: And what do you mean by that?

Rebecca: It’s not a good tea. It’s the first time I’ve ever had it. It’s supposed to be gingerbread. And I think it’s kind of disgusting. [LAUGHTER]

John: Now you know why we never get any sponsorships from tea companies.

Rebecca: I didn’t say where it came from. [LAUGHTER]

John: That’s true, but I’m sure there’s probably not too many gingerbread teas out there.

Rebecca: There shouldn’t be. [LAUGHTER]

John: In past podcast discussions with Jim and Sarah, we had heard about the writing group that you’ve all participated in. Can you tell us how this group got started?

Sarah: I guess I can take that one. I think it was about 2014…2015. Mike, do you know the year? He’s giving me a big nod. 2014. And Jim had recently invited me to write the Spark of Learning and his first series at WVUP and Mike was working on a book…had just come off sabbatical doing a cross country trip. And Jim was working on the first edition of Small Teaching. Since all three of us were working on book projects, Jim said, “Hey, why don’t we form this writers’ group.” And we started meeting. And it was really such a beautiful process to write my first book with my editor [LAUGHTER] reading every chapter as it came out, was a very developmental process. And as well as forming some really great friendships.

Rebecca: Sounds like a great start to a long endeavor. Can you talk a little bit about how regularly this group meets and if you still continue to meet?

Jim: We do continue to meet, but not regularly, as you might expect, [LAUGHTER] especially just because we were initially together at Assumption and were all working in the same area, and I just mean sort of geographical, and now Sarah’s in Boston, I’m going back and forth between Worcester and South Bend. So it’s a little bit harder to get us together in the same place at the same time. So we kind of meet on a more ad hoc basis at this point. if one of us has a project that we want feedback on will sort of send an email around saying “I need help, [LAUGHTER] and so let’s get together and can we have a writers group coming up?” So we’re still probably doing that maybe six to eight times a year, I would say. Maybe we were trying to do it every month but now it’s kind of morphed into that kind of more occasional kind of schedule.

Sarah: So we usually squeeze in a Christmas one.

John: That’s true.”

Sarah: So usually a mid December writers’ group. Some of them are in Zoom just because of how we’re spread around but the Christmas one is always in person. It’s really nice.

Jim: Yes, we have holiday themed groups. [LAUGHTER]

Mike: …with teas.

Sarah: Well, for some of us.

Mike: …or beers, but yeah, we’ve tried to have a couple of year that are also just social and in person. We’ve got more used to Zoom,as everyone has, and since we’re in three different…well, Jim and I live two doors from each other, but it’s harder to get us all together physically in the same space, but we’ve try to do that two or three times a year.

John:. When you were meeting regularly at Assumption University, what happened during the meetings when you got together? How long were the meetings? And what did you do in the meetings?

Jim: That would also depend upon where everyone was, in terms of their writing projects. A typical meeting would last maybe 90 minutes. Mike’s especially a social guy, [LAUGHTER] he would sort of want us to start with some social time, we sort of built that in. And so we’d have a little social time initially, you might be around a meal, for example, for the first 15 to 30 minutes, just catching up on our lives. And then we kind of just go through the project, essentially. So we essentially will just pick one off the pile, and say, “Okay, so here’s my essay this time,” like, for example, I might be doing a Chronicle essay or a chapter of a book. You have to read it in advance. So that’s an important thing to note, that we don’t read out loud, which sometimes writers’ groups will do that. You read in advance, we send the stuff out at least 48 hours in advance. You’re expected to read that material, come into the group having already written comments, either online or actually some of us still use the paper versions and bring those in as well. So then, essentially, the person whose material that we’re critiquing is supposed to just sit and ask questions, and the people that have read will kind of go through, essentially page by page, there might be like an initial comment, “overall, this is what I think of the piece,” but then we kind of go through page by page. That doesn’t mean every page is commented on, because it depends on where the issues are. But say, each person’s piece could take 15 or 20 minutes or so to walk through it. And then we move to the next one. And so we move around the circle that way. That’s the basic process, but what am I leaving out?

Sarah: No, I think that’s most of it… Kind of the flow.

Rebecca: Can you talk a little bit about the benefits of having formed the writing group and what other people might think about in forming a group?

Sarah: It is so invaluable, I think, in so many different ways to have first readers who can read something. It’s so vulnerable writing and putting your thoughts out into the world, and to have a safe place where people can give you feedback on very early drafts, I think is so valuable. And it’s also a really good test of are you expressing your ideas well, and are your references sound. I’m always telling Jim, people who don’t know sports aren’t going to understand this reference, and he does the same for my sci-fi references, and so that kind of tells you what a broader audience might think of your ideas. And it’s just a place to bounce your ideas around as well, like sometimes we do a lot of line-by-line feedback, and Mike is awesome at that from his journalism background. But we also sometimes just ask advice and just say, Is this even working? Should I draft this project and go to something else? Kind of really broad ideas as well. How about you, Mike, what do you see as the benefits?

Mike: The things you said. I think it’s a lonely pursuit, and if you don’t have an audience of people who are intelligent readers to let you know whether you’re totally going in the wrong direction or not, you’ve been staring at what you’ve been working on for so long, you convince yourself that yeah, this makes perfect sense, or this was a good move, or they’ll laugh at this, whatever it is. And so at some point, you got to try it out on audience members you trust and who also understand the writing process, because they themselves are writing and have written a lot. So they differentiate some things, maybe what not to do or to do. There’s a temptation in fiction writing, when I was in the fiction writing program, we were warned about trying not to turn the short story of the person being critiqued just into the story you had felt like writing instead. [LAUGHTER] Well, I figured out what the person is trying to do, and help them do that, and stay locked in on that, to some extent. So yeah, there was a lot there. And it is an extra deadline and something to get you moving.

Jim: That’s an important thing to know, too, that it was oftentimes a way of saying, “Okay, I have this book coming out, so I want to make sure I go chapter by chapter, and so I would make the writing group deadlines, my internal deadlines for getting those chapters done.” That’s a really, really helpful structural enabler, when you’re trying to especially do like a long project. The other thing I mean has come up in actually both these comments from Mike and Sarah, about the audience. And that’s true in terms of your audience. Think about your ideal audience who you’re trying to really write for. It’s also really true for your editors and potential publishers. And we often talk about that, like, “Okay, who would really be the right place to submit this to?” And we might be able to share examples of the publishers that we’ve worked with or strategies for reaching out to an editor, maybe we have a connection, and we know somebody. So we can have also that as well. So especially if you have books in your writers group who have published other things, you can start building networks around a writer’s group and I think that’s really important, too. Again, I think what Mike and Sarah are saying about audience is a real big benefit of you’re sort of thinking about the readers that you’re trying to reach, and you can often refine those. The example that Sarah gave about references, like if I’m writing to a community of people who love science fiction, I don’t have to worry about those references. But if I am going to expand beyond that audience then I have to either explain them or make more universal ones, and I think that process of like hearing at least two other readers, helps you expand a little bit in the number of people that you can reach with your writing.

John: And I would think having that relatively timely feedback would be much better than writing a big chunk of a book, sending it off, having it go out to reviewers and then getting reviews back when it’s perhaps a little bit late to make some of those major changes, that getting that more immediate feedback, I would think, would be helpful in keeping the writing project going in the direction you intended.

Jim: Absolutely. For example, I had like a little mini readers group with my Chronicle editor. Now I’d submit my work to her, and she would really sort of hack it to bits, [LAUGHTER] in a nice way, she’s a great person. But over time, I’ve sort of recognized, this is what she wants, this is what she looks for, this is what she changes. And I kind of learned to sort of mold myself into that kind of form that she wants. And the same thing is true for your writers’ group, you can sort of start to anticipate the things that they’re going to point out to you, which are often little habits that we have, which are not always great ones. [LAUGHTER] I always remember that Sarah would always point out that I used the word “so” all the time as like, “and so this,” and I never would have noticed that unless she started to circle all my “so’s” and then “look how often you do this,” that was like a really helpful thing for me to know. So now I’m aware of it, I still use them, but I’m just more aware of it now. So it’s very helpful to see other people’s perspectives and then you can start to anticipate that.Sarah sometimes uses phrase, “I had Jim in my head when I was reading this sentence,” or “I had Mike in my head…

Sarah: Yep. [LAUGHTER]

Jim: …telling this anecdote” or something like that. So that’s a really helpful thing to have as well.

Mike: Yeah. And it’s interesting, you’ll learn those things about each other, and then we can decide when to accept or not accept that, right? [LAUGHTER] In my case, and I think it’s true of Sarah, we can anticipate that if we throw in one too many anecdotes, we know Jim’s going to be there waiting to pounce on our extraneous anecdote that we love. And I’m somewhere in the middle, from the South, so somewhat more elliptical in my storytelling and less linear. So I’m favor of the extra anecdote. But there are times when you can tell, when you looked at it, and you have that voice and you go, “Okay, this really is extraneous, get it out of the way, don’t even bother other people with it.” And that’s that kind of thing and that’s kind of why we assign undergraduate students to do peer editing of each other is to learn to be editors, and to hear those voices too. So it’s really useful that way.

Jim: Mike, you know about other readiness groups and this question, John asked us about, like, guidance for other groups, can you give some examples, Mike, if you know of other writers’ groups that have sort of different ways of working?

Mike: There have been some groups that seem to exist just for people to pat each other on the back. And they don’t really do much criticism at all. A friend of mine who’s dyslexic said that he was in a group where you could never correct anything ever, grammatically or stylistically, which would drive me crazy, because I would want people to note every time I slipped up on something. So those are a couple of examples, and I’ve been trying to come up with some more, but it seems like there are a lot of variations in the rules in how they do it.

Jim: Sarah, you have also one that’s more like keeping you accountable, right?

Sarah: So I have another writers’ group with some friends at Tufts University, and we call it “writing and hijinks.” And so we gather together in a coffee shop or somewhere and we just write quietly, although one of them, like Mike, is very social and has trouble with the quiet part. [LAUGHTER] And so we just write, it’s just dedicated writing time for a couple hours, and then we go out and have fun. Although we’ve been sliding more and more towards just the fun and not doing the writing [LAUGHTER] over the last few years. This is motivating me to say to them, maybe we should bring back the writing part.

Rebecca: We have a writing group at Oswego that functions like yours, Sarah, that you’re talking about, where it’s just kind of some dedicated time, accountable time, this is the time we’ve set aside to write…

Sarah: Right.

Rebecca: …which can be really helpful if you have a designated time and space to do that, if you aren’t good at scheduling it on your calendar otherwise. Julie

Jim: Joli Jensen has a book about writing from the University of Chicago Press, and she actually makes the argument that writing groups for academics should not be content critique groups, which is what we have, actually. And she makes the argument that it’s not helpful, for example, English professors to be working with a psychologist because they won’t get your references and all that kind of stuff, and it won’t be very helpful to you. But actually I would argue that what we’re doing here, because most of us are running for, like broader audiences, the content critique part is really important. If Mike and Sarah don’t understand what I’m saying, well, then I’m not gonna be able to reach a broader audience instead of just writing specific to my little expertise audience. So that’s another way to think about distinguishing content, whether you focus on the content or just the writing, the sort of grammar and the stylistic part of it. So that’s another way to think about what are you actually trying to do in your writers’ group? Are you trying to expand beyond your discipline and getting different kinds of feedback? Or are you just trying to focus on, as Mike said, maybe patting each other on the back or accountability or style and grammar? So there’s a lot of different ways you can go.

John: I think that would depend a lot on who your intended audience is. So that if you’re all psychologists, or all economists, or all mathematicians, if you’re going to be submitting things to journals, it might not be a bad audience. But for a general audience, having someone who is not necessarily an expert in the field, seeing it from a different perspective would lead to a lot of really productive comments in terms of what some of the hidden assumptions are, the implicit assumptions in the writing.

Jim: Absolutely, you can actually even think about it as a teaching thing too, I mean teaching to like a gen ed class, their first-year students, that’s almost like a content critique group in the sense you can’t take too much granted about background knowledge in discipline, like a really focused content critique group, all from a discipline, you’re essentially teaching a graduate seminar. And so you’re looking for different things from those experiences, and you’re being required to explain different levels of things to those two audiences.

Rebecca: So I’m curious what participating in this particular writing group with the three of you has done for you as writers? How has it shaped your writing? And why are the three of you a good group for each other?

Sarah: I think that one of the things that has really shaped my writing process, this is beyond all of the wonderful things that we have said already about audience and things like that, is being in a writers’ group with two English professors is really valuable. I tested out of intro writing in college, so I actually never took a writing course of any sort, and then decided I wanted to be a writer. And my writing is very organic and intuitive. And working with these two really helped me like the things that they would correct. Like I didn’t know periods went inside the quotes, until Mike corrected me. [LAUGHTER] But the first year, we were writing together, and you would think I would just absorb that from reading, but I just never noticed that. And so really tiny things like that, but then also just thinking about the structure of your writing and the organization of your writing, because I had never really done that for this kind of general audience writing. And so having this writers’ group with writing professors is really amazing, as Jim would sometimes send me to the whiteboard, when we would meet [LAUGHTER] at Assumption, because it’d be like, you’d have a lot of really interesting ideas here in the writers’ group, but they’re just in a big pile,[LAUGHTER] and he’d have me map out, do a literal map of my work. And that was very, very helpful.

Mike: That’s one of the fascinating things about being in a group is working with two people that have such different composing processes. And Jim, you’re fairly linear…

Sarah: fairly…

Mike: kind of have it in order [LAUGHTER] and smoothly.

Sarah: …the computer. [LAUGHTER]

Mike: So what was the computer program that you’re using that divided the screen into quadrants?

Sarah: Yes, Scrivener, you can put your bits of writing on literal digital index cards, and just move them all around.

Mike: Yeah, which is something I’ve seen poets do. I’ve seen poets tear their poems apart, line by line and move the lines around in the coffee shops. So it’s just two equally effective ways of doing that. And I guess I’m somewhat in the middle, but with the road trip narratives, obviously, there was someone’s this town and that town, but it’s fascinating to watch how people’s brains work differently.

Jim: Yeah, I think for me, Sarah and Mike are different kinds of writers. But as a result, I’m kind of benefiting from both the sort of storytelling advice that comes from Mike and from Sarah, again, more of the research, making sure that I’m supporting my points well, and thinking about other perspectives on the topics I’m addressing. And it’s helpful to get both those things, because I’m getting different things from others, like editors, for example. I like the idea of starting a piece of writing with a story or something like that, my editor at The Chronicle always wants me to lead with the thesis, essentially. And so I’m trying to put some of Mike’s ideas advices into that kind of storytelling opening, but also trying to satisfy my editor. So anyways, it kind of helps you develop new strategies and see how strategies might land with different people and Sarah’s especially good at ways that I should include more readers, more thinkers from different backgrounds, and essentially making my writing more inclusive. And I think that’s really valuable as well, and making sure that I engage with the research that I might not be aware of, in the teaching and learning fields. So that’s really important, too. So I can look at my own writing and see how it’s benefited from both their perspectives.

Rebecca: I think one thing that I’ve benefited from when I’ve participated in writing group opportunities is actually seeing early stages of other people’s writing, and just what that looks like and feels like and sometimes provides strategies to future me [LAUGHTER] when I’m stuck, [LAUGHTER] like, “oh, wait, this other person would have attempted it like this.” And when I felt stuck on something, sometimes I use the strategies that I’ve observed other people using, which can be really beneficial, that things that I wasn’t taught or hadn’t been exposed to otherwise.

Sarah: That’s great.

John: You mentioned future me. And one of the issues that we know from behavioral economics and psychology is that when given a choice between pursuing your long range objectives and immediate gratification, immediate gratification sometimes wins out, [LAUGHTER] because it’s always easier to start these things tomorrow or next week. And one of the things you mentioned earlier, Jim, was that having the regular meetings of the writing group provided a bit of a commitment device that you want to get to the meetings with some new material you want to share that material before it. Do you think that has helped increase the pace in which your work has taken place?

Jim: For me absolutely, yeah, having that structure there. I think structure is always a really helpful thing in teaching and writing. If you sign a book contract, the deadline might be two years away. So you know that two years, an editor might check in after a year, but like, you need to have more structure in that to kind of get yourself moving along. And I’m a pretty structured person, but I can also fritter time away. I’m pretty good at that, too. [LAUGHTER] But I’m self aware enough to know I need a little help in terms of structure. And so it definitely has pushed me to become more productive.

Sarah: Absolutely. I agree. This whole conversation is making me want us to do this more regularly, [LAUGHTER] instead of the as needed, because I think about especially the beginning of the writers group. And when I was writing Spark of Learning, my daughter was a toddler. And she was only in daycare three days a week, and I was on the tenure track and just doing a billion different things. And I do not think I would have written that book if it weren’t for writers’ group monthly meetings. I think at that point, we were really regularly meeting in person monthly. And just finding the time at four in the morning [LAUGHTER] or terrible times often, but I don’t know that I would have gotten it done if it were just like, “oh, I have a contract and I must do this giant thing.”

Mike: That definitely helped this begin…the structure. I don’t have a long-term project I’m working on currently so hard for me sometimes to come back around and really have anything that’s worth them looking at. But yeah, you mentioned those long period of the book contract, and there’s a long period of only doing shorter things, whereas so there’s not that continuity from chapter one to chapter two to chapter three, and you’re eager to write those next two pages that are the bridge to the next thing, so group can help with that as well. It seems like a lot, but what happens in writing group is when someone is working on a Chronicle column that needs to be turned in in 48 hours or letters and things like that. And I was wondering if you can give examples of that kind of stuff when the group comes into play. That’s not the big primary thing we’re working on, but it’s kind of the emergency stuff.

Jim: We’ve done a little bit of that, but it’s kind of hard because especially for an upcoming deadline, I think typically we’ve handled that when someone just needed something very quickly, we just handled that by email. But the writers’ group is available in those emergency situations, because we already have a quick sense of like, if Sarah sends me something, and it was for a quick turnaround, it’s for The Chronicle, I can be able to spot very quickly a couple of things that you could maybe do in that short window. And likewise for me. So it’s challenging, though, because you want to have time for people to have opportunity to read and think about that as well. But it’s available for those situations. So it’s good for that.

Rebecca: I can imagine that you’re strong relationship with one another, having written with each other for so long allows for that quick turnaround by email, because you can sense the tone someone’s providing some feedback in [LAUGHTER] and it…

Sarah: Help!

Rebecca: …has a different kind of context. [LAUGHTER] Yeah.

Jim: Absolutely.

Sarah: And it’s such a pleasure, like it’s hard to ask for emergency work from people, even people you’re close with, and maybe even especially people you’re close with, because you know, they have so much else going on. But it’s such a pleasure to read these gentlemen’s words, that it’s not a burden at all, even the emergency ones because it’s like, “Oh, I get to relax, not with a cup of tea, [LAUGHTER] and read some words from Jim or Mike.

Jim: That’s true. That part of it is good, because you know that you can help them in the emergency because they’ve helped you in the past, that same situation. Right now, you know, I’m sort of developing a substack account. And I haven’t really used the writers’ group for that. And as a result, I need to have my wife proofread for me,[LAUGHTER] because substack doesn’t have an editor, and I’m really annoying her by this process. So I need to start using my substack columns for writers group stuff as well.

Mike: Yeah, it is really helpful and we know that there’s just a lot less explaining to do amongst the three of us. If I were going to send something cold to another friend, I think I would have to explain a lot more about what was going on for them to get it. So that’s incredibly useful that way. But it’s funny how different it is from like growing up in a newspaper family, and I would interview someone in the morning and I would revise it four or five times in a period of like four hours and hand it to my editor who would then tell me to change three things. And the whole thing from beginning to end was less than a day. But there was no deep thought, there was no deep reflection or big structural things, and then my time sort of expanded magically as I became an academic and suddenly you can get trapped in a lot of whirlpools of uncertainty about which way to go or that way to go. Whereas in journalism, there wasn’t much choice a lot of the time.

Sarah: Writers note, I love that “whirlpools of uncertainty.” If I were in writers’ group, I would say underline it, put a little exclamation point next to it. [LAUGHTER]

Mike: Well, I was a really terrible student in an undergraduate poetry workshop at the same time my sports editor was trying to teach me to write in a linear way. And my teacher said at one point, Mike what you’re writing, as poetry is prose with line breaks in random places. And he said, “Let the language go where it wants to go.” And I was having a hard time imagining telling my sports editor that on [LAUGHTER] Friday night during high school football season, but it seriously took a long time for me to learn I could do both. I could loosen up and be this one kind of writer, and then be this very mechanical quick writer. And that took years as an undergraduate to figure out how can I do both of these things at the same time.

Rebecca: One thing I heard Sarah say is that currently you’re meeting as needed, but then I also heard in the conversation, that maybe we need these things, and we just don’t know it. So having it regularly on the calendar could be helpful. [LAUGHTER]

Sarah: Mm hmm.

Jim: Yes, we’re convinced now this conversation is very helpful to us. [LAUGHTER]

Mike: Yeah, and plus this almost the end of the semester, so suddenly, we’re gonna have a lot of time on our calendars.

Sarah: …well, those of us still teaching. [LAUGHTER]

Mike: Yeah.

Sarah: I’m at 12-month contract now.

MIKEL Oh, you are?

Sarah: Yeah.

Mike: For you, year round.

Rebecca: Me too, Sarah. [LAUGHTER] Maybe we need a separate writers’ group. [LAUGHTER]

Sarah: Or therapy. [LAUGHTER] …support group.

John: What sort of advice would you give to other people who are thinking about forming a writers’ group?

Jim: Actually, there have been been other people in the writers’ group over the years, I’ve been in other writers groups too. For me, the thing that I look for is people who I thought were sort of ambitious and were writing, because you don’t want to be the person who is trying to gather people together, when they’re just not writing, they don’t want to write, you don’t want to be like the taskmaster in a writers’ group. So find people that are writing and are interested in writing, and find people that you find interesting, you want to hear more of their thoughts. So that has always been a driver for me. And sometimes, if I’m looking for people that I want to hear their thoughts about my own writing, and I want to talk to them about their own writing, I’m gonna be more interested in going to that writers’ group and participating. And so those are sort of two general points I would make: look for people that are writing and interested in writing, and then people who are interesting, you want to see it develop and getting access to their ideas as they’re developing. And I would just say, most people, I think, are flattered, you would like to hear and read their work. So if you identify someone that and you might say, “Well, oh, you know, they’re probably too busy, or they probably already have other people to work with,” that might not be true. And so extend the invitation and see if they want to join you in pushing writing forward.

Sarah: And I think I would add, it’d be hard to figure this out, [LAUGHTER] but one thing that I appreciate about this group, and I would want in any writers’ group is having a similar level of like sensitivity and openness to feedback and criticism. So I think that all of us are really great at taking feedback and taking constructive criticism, without either taking it personally, or I think Mike referenced this, but sometimes the Jim in my head or the Jim on my page, I’m like, “Yeah, I’m not listening to you.” [LAUGHTER] But then a lot of times you do. And so t o people who are at that level of like, open to hearing criticism, but also confident enough to stick with their gut if they feel it’s right. And yeah, and so that’s a delicate thing. And it’s hard to figure out about someone without actually engaging in writers’ group. But I think if they were in your department meetings or on committees with you, kind of keep an eye and get a sense for how open they are to other people’s thoughts and how willing they are to give them.

Mike: Yeah, and along the line with open is something I always warn undergraduates about, but in grad school it would come up with separating the characters on the page, even the narrator, that that person I’m writing the first person about, it’s Mike, the narrator on that page, but it’s not necessarily all of what I am. So, ‘cause when you’re talking to someone, then you can point out stuff that you think is maybe sexist, or is whatever, but be able to talk about it in a separation where you’re not personally attacking the writer, but that you are holding that in a certain place. I think we’re really good about those sorts of things, we know the level each of us are trying to get to, we’re just trying to help each other get there and again, smooth out anything that the audience might misinterpret. That’s a mental discipline that takes a while to be learned. And I’ve had friends who have been in groups where things have devolved to name calling, and to really unfortunate situations where the writing group had to be broken up, and so I’ve even heard that happen in academic settings, like in actual seminars. Fortunately, I was not part of any of those, but where people went stomping out of the room. And so yeah, having the maturity in just the way you carry yourself in life, I guess, to know how to treat other people and work with them constructively, a really important thing. But otherwise, in terms of advice, let’s say that whatever kind of writing group you want to have, you want to sit and just right beside each other and then go have drinks that what this one’s gonna be, are you all aiming to publish and you’re trying to help each other get published? That can be a very different mentality, depending on what you’re trying to do. That’s my two cents.

John: Do any of you have any other thoughts about writing groups that you’d like to share with our listeners?

Jim: I will say just as an editor, you know, editing the book series, I can also sometimes tell people who had writers’ groups when they turn their manuscripts in who had prior readers. So just from my perspective as an editor, I absolutely recommend this, forming a writers’ group, especially if you’ve signed a contract, and you’re intimidated by it. The fact that you have this deadline, you have this big project coming up, A writers’ group can not only put some structure on to the process but also forestall a lot of the work that I will have to do as an editor. So you can make my life easier when you’ve had some prior readers who kind of work with you to iron out a lot of the kinks that have developed, as any writing project will have in the drafting.

Sarah: Yeah, my editor was very grateful for writers’ group. We met monthly, my editor for Mind Over Monsters and she was wonderful. But writers’ group was like a third character in the room. [LAUGHTER] Because we would meet monthly, I’d be like, “Alright, well, here’s the chapter, but this is what writers’ group says, and they think that I should reframe this.” So it was really like the three of us, or I guess, the four of us, melding you two into one being in the conversation and in the room. And that was an interesting process.

Mike: I think another thing that’s been interesting to me is that because Jim and Sarah have books out a lot of the time, and there’s a good bit of talk about the publishing aspect of it, there are times when say, Jim might know someone that one of us should hit up, or contact, or things like that, or how to interpret an editor’s reaction. And there’s been a good bit of conversation about those things where I kind of felt like I know, these editors I’ve never met. [LAUGHTER] I’m sure that’s a big stopping point for a lot of people is they’ve written this thing, and now they don’t know what to do. They don’t know what the path is, and how to get there, how to get someone’s attention. And so there’s a lot of talking back and forth about things like that.

Rebecca: Do you have wishes for your group moving forward?

Jim: I guess, more regular meetings.

Mike: …more regular meetings, yes. [LAUGHTER]

Sarah: Sounds like a way to save Jim’s marriage.[LAUGHTER]

Mike: I guess, I should sell them another road trip so I could write another road trip manuscript.

Jim: Yeah.

Sarah: There you go. I would read that.

Mike: As we sneak toward possible retirement…a travel book…that could be the plan.

Rebecca: So we always wrap up by asking: “what’s next?”

Sarah: Well, more regular writers’ groups, I guess. But I think as Mike just referenced, he’s retiring. So he’s going to have a lot more time to write coming up. And Jim and I are both, I think at the phase of thinking about next book projects. And so maybe it’s going to be a new day for writers’ group.

Jim: That’s true. I’m actually doing the final, final, final revisions of my current book manuscript, which are due on April 22nd. So that will be coming up shortly. And so I’m in that phase where like, I’m kind of finishing everything up, and also the editing and all that stuff, and copy editing, but still at least starting to open up in the sort of creative part of your brain about like, what might be next? And so that’s maybe one final thing I might say about writers’ group is sometimes we will come in with just like a one pager of ideas. And Sarah has done that several times. And we kind of give her feedback like, this is the one that seems like it might be the best path to pursue, for example. Anyways, I’m in that space right now. So I hope that for me going forward writers’ group might help me land on the right project next.

Mike: I should mention too that just the venting of frustration is good. And it just crossed my mind, one of Jim’s funnier Facebook posts of all time was when you were finishing up a book and what you posted about how you felt about the book. I don’t know if you recall this or not?

Jim: Yeah, absolutely.

Mike: So tell them. [LAUGHTER]

Jim: So I just remember talking about the fact that whenever you finish a book, I’m just so sick of it. At that point, I never want to think about it again. [LAUGHTER] And then of course, a year later it comes out and you have to think about it all the time. [LAUGHTER] I’m a person that has multiple interests, and they sort of change on a regular basis and like, by the time I finished a book, I’m like, “Okay, I’m done with this now, what’s the next thing?” But then, you know, hopefully a year goes by when it’s coming out, and then you get interested in it again during that time, hopefully.

Mike: Well, the specific line I remember was, “I hate, hate, hate, hate, hate you, stupid book.” [LAUGHTER]. It was that quote.

Jim: Yeah.

John: Well, I can see it’s probably good that you focus on the writing and not the marketing of these books. [LAUGHTER]

Jim: Yeah.

Mike: It wasn’t this book, it was a different book.

Rebecca: Well, thank you so much for joining us. It’s always a pleasure to talk to you about your work. And we’ve been hearing about this writers’ group [LAUGHTER] for a while. So we were glad to have the opportunity to dive in a little more deeply.

John: And we’re also grateful that we’ve been able to benefit from the writers’ group. I was just thinking back, four of our reading groups over the last few years were products of this writing group.

Jim: Wow.

John: So we appreciate the work that you’ve been doing.

Jim: That’s very cool.

Sarah: Thank you. It’s an honor.

Mike: Yeah, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

342. Infographics

When papers and projects are due at the end of the term, students often procrastinate even when the projects are carefully scaffolded. In this episode, Michelle Kukoleca Hammes joins us to discuss how a series of infographic assignments, combined with peer and instructor feedback, provide an engaging and productive learning experience. Michelle is an associate professor of political science and a CETL Fellow for the Center for Excellence in Teaching and Learning at St. Cloud State University.

Show Notes

Transcript

John: When papers and projects are due at the end of the term, students often procrastinate even when the projects are carefully scaffolded. In this episode, we discuss how a series of infographic assignments combined with peer and instructor feedback provide an engaging and productive learning experience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michelle Kukoleca Hammes. Michelle is an associate professor of political science and a CETL Fellow for the Center for Excellence in Teaching and Learning at St. Cloud State University. Welcome, Michelle.

Michelle: Hi. Nice to be here.

John: We’re looking forward to talking to you. Today’s teas are:… Michelle, are you drinking tea?

Michelle: I am. I don’t always drink tea, but I made sure this morning I did. And so I’m actually just drinking a green tea, a milk tea. It’s not quite as good as others I’ve had. I have a colleague from Wales who makes the best green tea, but it’s good this morning.

John: …and Rebecca?

Rebecca: I have a Hunan Noir Tea, which is very tasty.

John: You left out the Pinot.

Rebecca: I know. [LAUGHTER]

John: Okay. And my tea is a black raspberry green tea.

Michelle: Sounds nice.

Rebecca: So we’ve invited you here today, Michelle, to discuss how you’ve been using infographic assignments in your online comparative politics classes. When you first started teaching this course, what kinds of assignments did you use?

Michelle: I use a variety of assignments. I’ve been teaching this course for 27 years now. And then this is mainly for my intro level comparative course. But I’ve adapted it for upper levels too, so mainly traditional assays early on. As I moved to the online environment, I would still have them do a major paper of a country study. And so pretty traditional assignments for the most part, but I wanted to make some changes for the online environment especially.

John: So what made you decide to change to infographics?

Michelle: Well, a few things. One is I felt that students were doing these papers that were quite lengthy in their country studies, and they would turn them in at the end of the semester. And we never really had a chance to talk about the elements of them in depth prior to that. And so I really wanted to change that. So it wasn’t just the end of the semester. And they weren’t sharing them with each other, because of the format of them as just being a lengthy paper, it wasn’t the kind of thing that they would be interested in exchanging with each other. And then frankly, I was getting a little bored, [LAUGHTER] the format, not so much the content or their insights, but the format just became very boring. We all know that when you’re working with one individual paper, it can be really exciting to have that dialogue with the student. But when you have a stack of papers, it suddenly becomes too daunting. And so I was just looking for a way to make us all happier, actually.

Rebecca: That sounds like a good motivation. [LAUGHTER] Can you talk a little bit about what the infographic assignment is?

Michelle: So I will talk about in terms of that intro level comparative politics class. One of the things that there’s not time to do in any semester is to really cover every country in the world, obviously. First of all, I wanted to make sure that students could pick their own country that they have an interest in, and that would broaden the number of countries we would cover. And then I switched it to infographic for the online component because there were a few things I wanted to do with it. One is to retain the goals I had in mind for them learning about a country. But also then I wanted to tie in some best practices for online presence in the classroom, for example. My discussions, I was also unsatisfied at the beginning, we learn as we go along, and my discussions tended to also be maybe a little stale, or I’d throw out a question and I would get very similar answers from everybody. They were going back to the textbook and giving me something rather than being creative or energetic in their own answers. And so the way the infographic works is I have about 10 through the semester that they have to do. And so it’s basically breaking up the country study into smaller pieces, so that they have different topics as we go along the different topics in the class. And then before they submit them to me, they submit them into the discussion for the appropriate week. And that way, all the other students can see them, it gives them a lot of different skills. One is that they are able to critique other people’s work, they are gaining more knowledge about a variety of countries. It allows them to practice skills on their own in terms of working together. So I don’t see these always as only individual assignments, although that’s how they’re graded, but kind of a group work assignment that they can all enhance each other. And so there were a lot of ways in which I wanted to bring different elements into the same assignment, particularly because I didn’t want to have to think of how to bring these elements in and do different assignments for each thing I wanted to bring into the classroom. Having one assignment that was consistent really helped them.

Rebecca: It sounds like by having a consistent assignment, you do have to learn how to do the assignment over and over again, that part of it is covered. You do it one time and then it’s clear how to do the assignment and then you can focus on the content and the material.

Michelle: Yeah, exactly. And one of the other things is, unlike just having one country study at the end, there was a lot more formative assessment that could go into it. So exactly that, where we’re able to help build skills up early on. Some of the content early on is pretty simple. The first one is making an infographic about the demographics of the country. And so it’s pretty simple, but it allows them to get the first skills if they’ve never used, for example, Canva, or other programs that allow them to make an infographic. So they get their feet wet with that. It allows them to think about how to communicate in a different way other than a paper, because I also began to think about my intro level students who many would become political science majors, but many wouldn’t. And so in a course that they were using for general education, they’re not going to be writing political science research papers. And so allowing them in different ways to express themselves I felt was another skill that they could bring.

Rebecca: It was nice to see this scaffolding, separating out some of the technical skills from the content, because often we’re doing all of those things at once. And it sometimes can be unclear whether or not it’s a technical skill that is lacking or the lack of understanding of something. So separating those out of it really does seem like it would help have a clearer sense of where students are. And, as we know, [LAUGHTER] if you try to ask students to do too many things in one bigger assignment, then certain things end up edited out. And that might be the thing that was really critical. So it’s nice to have those separate assignments to focus in on those.

Michelle: Yeah, exactly. And I really enjoy the fact that, rather than focusing on grading a long paper, I’m able to go through that process with them. And then also in discussions, it gives me so many opportunities to re-teach something. So if I’m noticing in someone’s infographic that they’ve made an error, or they have a misunderstanding of a concept we’re working with, then all the students in the class can see me correcting that and doing it in a gentle way, that seems a little bit less high stakes for students that because it is just this one assignment, and I’m making this one little correction here, but I think it benefits everybody.

John: So for those students who haven’t done much work with infographics or with graphics in general, do you provide any instructions or recommendations to them? You mentioned Canva, do most students use that? Or is it a mix of tools?

Michelle: They usually do end up using Canva, because that is one of the tools that I put in the directions. And so I have quite lengthy directions in terms of guiding them through accessing technology, allowing them to know that they can use various different technologies, depending on their accessibility to them. There are some really great programs that of course, cost money or pro versions. But I let them know that for our purposes, they can certainly just download a free version of a program. They could really even do it in the Microsoft Office Suite in a variety of ways if that’s where their skills lie. And of course, that’s something that all the students have access to on campus. And so I do get them lengthy directions, they’re not all my directions. I don’t feel like I need to reinvent the wheel. So I just include several videos from YouTube that talk about how to use Canva. And those are sufficient and actually probably better than I can do. And so I put those out there with other directions, I give them alternatives. And then of course, if they’re still stuck, then they can always work through it with me. We can have a Zoom meeting where they pull up their materials, and we work on putting it together.

Rebecca: You mentioned that students were sharing infographics with each other, it sounded like maybe through a learning management system or something. Can you talk a little bit more about what that sharing looks like and the kind of feedback that students are giving one another.

Michelle: So the learning management system we use here is D2L Brightspace, and there is of course, like in most learning management systems, there’s that built in discussion feature, so I just use what’s already there. I don’t mind using third-party applications., but it just tends to make things trickier. So I just use the discussion. Have them post it right to the discussion, and have them use the discussion tools, whether it be just typing in or using some audio or other tools to give feedback to each other. And I find that the discussions are so much more lively. Even more than I expected. The first time I used it, it was for a summer class and I thought “okay, I’m gonna try this now because it’s summer, and no one wants to be sitting writing a 20-page paper in summer. What am I going to do?” So I put it out there for the summer class and I thought, Okay, well, maybe they’re busy with other things or they’ve got excitement in their own life over the summer or they’re working a lot of hours. I had never had such lengthy discussions go on in any class before that. They really enjoyed looking at each other’s work rather than just from me. Many of the students chose countries that reflect an interest they have, maybe it’s somewhere that they’re actually from, although I usually try to encourage them to broaden a little bit. But that also brings such a benefit to the rest of the students. So we do have students who do that. I have students who maybe have gone on vacation somewhere or have a family connection somewhere, I’ve had some that have been stationed in the military, and come back, and they want to talk about what they noticed when they were there, so they put a lot of that in the infographics. I found just use the basic tool, but allow it to be opened up for them. And I also try in discussion, and I do this even in other discussions that I have online, because I usually try to hang back for the first bit, quite a bit. I allow that to be organic. And then I’ll go in and make sometimes just a summary of everything that’s been said or things that I need to correct. So that it’s not simply another lesson from me. But I have that ability to correct when I need to.

John: You mentioned the first infographic assignment had them work with demographic data and display that, to what extent are these assignments tied to the specific course work that you’re doing during each of the 10 assignments?

Michelle: For most of those 10, they’re directly tied to the material for the week. So for example, the textbook will have a chapter where we’re talking about government structures, presidential systems, parliamentary systems, et cetera. And so the infographic for that week will be directly related. So they’re going to do an infographic laying out the government structure. And then another module will be on political parties and how they work in various parts of the world. And so it’ll be directly tied to: “So show me your country, and show me the main political parties in your country. Tell me what their main ideology is. Tell me who’s in power now… which party? And what does that mean for the policies that get put forward?” Another one would be one on current events, the last one that they do is: “Well, what’s going on there now? …and “With everything you know about your country now, what do you predict for the future for them? How do you think they’re gonna solve this?” So a little bit more policy oriented… also, “look at what they’re actually doing in terms of legislation and implementation.” And I also do one… I don’t use the textbook anymore, but it’s something I used years ago that I retained the content in a module, which is to look at the concerns of any country in terms of every country’s concern with prosperity, security, and stability. And so I asked them to just put out an infographic that asks them to assess threats to those things, and looking at their country. And so it’s a little bit on the current events side, but it’s focused for them in terms of every country needs to be aware of its security… what’s happening?

Rebecca: You’ve hinted that students are really engaged in these assignments because of discussion forums. But can you elaborate on just the general sense of what students have been able to accomplish by shifting to this format, and also their engagement with one another?

Michelle: I think, several things, some of which I’ve mentioned. So the formative assessment part, I think they gain a lot from not just getting comments at the end of the semester, oftentimes handed in that last week. And then, frankly, I don’t expect that every student has even read my comments at that point. So just having the opportunity to fully engage throughout the entire semester, and so we’re not also dealing with, “Oh, I have three days before this paper is due. So let me think about it today, make a plan to think about it more tomorrow, [LAUGHTER] and then maybe I’ll do it the next day before it’s actually due.” And not that that’s all students, but I know when I was an undergrad that happened to me. So I think that just slowing them down, allowing them space to actually think about their country in multiple ways, and not have it be “Okay, we’re all working to doing your own country study at the end of the semester,” but having them actually engaged the whole time, I think is of great benefit to them. And I think they learn a lot more. I think that taking it in these smaller chunks, actually has made them research a little bit deeper on each of the topics because they’re not overwhelmed with “Okay, I’ve got all of these topics now to cover in this country study.” And so they tend to just do it in a way that is most efficient for them, let’s say. And so I think this gives them that way to back up a little bit and actually enjoy the process of learning rather than the process of an assignment only.

John: I think you’ve made a really good point there about students’ tendency to procrastinate. I was thinking back to when I was an undergraduate, and actually a graduate student as well. And I can’t recall a single time when I didn’t start writing the paper the day before it was due. I spent a lot of time gathering materials, taking notes, and organizing and putting an outline together, but the writing generally took place in just one day. So, [LAUGHTER] this is forcing students to engage in a little bit more spaced practice, to engage with the material regularly, and I think there’s a lot of benefits over that rather than the traditional way of just rushing through and getting it done right before it’s due.

Michelle: Yeah, exactly. And I’ve adopted that in other classes too, the idea of doing a lot of prewriting throughout the semester. So I’m a big believer in the prewriting process, and it’s very valuable, and it is doing work. But when I mentioned earlier, sort of they spend the day thinking about it, I’m thinking more of myself as an undergraduate thinking about not the assignment, but thinking about the fact I have to do an assignment.[LAUGHTER] So it’s more anxiety, not so much productive work on the content of what I’m doing. And so yeah, the prewriting work for any assignment is really great. So I’ve been trying to shift that mindset. I also teach a research methods course in my department and I teach our senior seminar. And so the senior seminar, especially, just making it about an entire semester-long process, I think, has taken a lot of pressure off of them. They seem a little relieved by the end, it’s not as daunting,

Rebecca: It definitely seems like it deepens the study and the engagement, not just because it’s broken into smaller pieces, but because it’s a consistent country the whole time and because it’s a consistent kind of assignment the whole time, it really provides a structure to be able to do that. I’m curious whether or not any of your other colleagues have also shifted to doing assignments in this format, or other similar kinds of strategies.

Michelle: I can’t really say. I know that in my department, we’ve talked about them a lot. And so I think people are adapting parts of them. I don’t think anybody does it the full way that I’ve described in terms of the full semester, but we have talked about just different ways of communicating. I think that’s going to shift again, obviously, AI is something that everybody is grappling with how to best use. So I think there’ll be some shifts coming. But I can tell you that I was Interim Director of the Center for Excellence in Teaching and Learning, and now I’m a faculty fellow there. And I’ve talked to a lot of people across campus, and some of our off-campus partnerships about this assignment, because I get really excited about it. If you haven’t noticed, this is my favorite thing. And so I’ve talked to a lot of people, and I just can’t be sure how many have actually applied some of these things. But they certainly seem interested, and they’re certainly searching out the same types of things I was searching out: how to make things, first of all, more authentic for the students, how to engage the students more with themselves in an asynchronous class or with other students. Even in a synchronous class or on-the-ground class, you could still have the same assignments. You might want to introduce them differently, you might want the discussions to take place in the classroom, rather than obviously online, because you could engage in person that way. So you can do kind of a round robin around the room, maybe… any of those techniques. But I also let the students know that, for them, they will become a mini expert by the end on that particular country. So they should all understand the topic areas but. becoming an expert on the individual countries that’s all their own. And so I hope that colleagues, no matter what field they’re in, can find some way to utilize that. Otherwise, they’re probably just a little sick of hearing me talk about it. [LAUGHTER]

John: Speaking of fellow faculty, has anyone else shifted to using infographics at your institution as a result of your work with us?

Michelle: Yeah, I can’t be quite sure. But I do think that in talking to a lot of faculty that they are shifting modes of communication for students, and I have talked to people who are much more expert at creating infographics themselves. You can do it very, basically, but you can also get really creative and high level with it. And so I oftentimes collect infographics I see for other fields, so I share those with faculty. I’m not sure. It’s one of those things that I so appreciate a podcast like this because sometimes teaching can be a lonely endeavor. We’re a little bit maybe shy about sharing what we do, because we’re afraid it’s not measuring up necessarily to what others are doing. So it’s a good question, though, and I think that that’s something that maybe I want to engage… maybe in a workshop in the future on campus and bringing faculty in and seeing who’s doing this type of thing, or maybe another brand new idea that’s even better.

Rebecca: I think one question that often comes up from faculty when we’re talking about kind of alternative formats or unessays or other [LAUGHTER] ways of not necessarily doing a more traditional essay kind of assignment, is how you evaluate the work. So I wonder if you can talk a little bit about how you’re evaluating these assignments, just to give faculty a sense of what that might look like and what that feels like in comparison to a more traditional kind of assignment?

Michelle: Yes, that’s a great question. I’ve talked in many ways about why I did these assignments in terms of just making it a more engaging process. But in terms of measurable, assessable goals, I do use rubrics, and have laid out exactly what skills I’m assessing. So I do have a section of the rubric that is based on the skills of communication. I have a very low stakes, low number line for things just like visual engagement, because I don’t want it to be students worried about the visual engagement when they need to learn the political science concepts. but I want to let them know that that’s important. So I have in the rubric descriptions in each of the columns about visually engaging: Does it bring the reader in? Is it easy to read? So I have that skill section, I have a section specifically on the content itself: Do they understand the concepts? Are they using the language that we use in political science? And then I also have sections for pretty typical traditional looking at their writing and seeing if it’s appropriate writing levels, tell them take a look at other people’s infographics… go out in the world and look at the bottom, there will be the sources, you need to include those, you need to not leave those off. So even practice that they would get in papers about citing things, if they put a graph in, you still have to say where this graph came from. So the practice that they would normally get are the same skills. And so I just lay it out in a rubric for them, as I do with most of my assignments. Of course, the rubric is not secret to the students, that goes out with the very first assignment: this is what you should be looking for. I’m going to use it for grading, you can simply use it as a checklist. You can use it to evaluate other students and give them feedback and tell them where you think there could be improvement. And so I think that the assessment itself is pretty typical to what I would normally do when thinking about an assignment: What are my goals for the course? How does this assignment help to meet those goals? And then how do I convey to the students whether or not they’re meeting those goals? Or essentially where they’re at in the process of becoming expert in that goal.

John: Do you use one rubric for all the assignments? Or are there separate ones for each of the infographic assignments?

Michelle: So I’m a big believer in constructing the outline once and then using it, but having different content in it. And so I use the same basic rubric for each of the assignments. And then I give the feedback in that content area really is the main portion. In the content area, I will make some changes to what they need to do specific for that assignment. So if it’s political parties, I would have expectations about what types of things they’d understand about political parties. And so that will change but my basic outline of what that looks like. And I’m a big believer in that kind of consistency in everything. So I can tell you I do that also with just how I set up the modules. And so my modules always have the same outline. There’s an intro with the learning outcomes for that specific module: what can you expect to learn this week, then the section with textbook readings. So this is what you’re going to dig into and read in a very traditional way. Then there’s a whole separate content section, which might be a video I’ve created, it might be videos that I found online that they should be engaging in. And then I usually have a pre-test and post-test quiz, and then the assignment for the week. And in this case, always associated with that assignment discussion. And then always at the end, I have a section, another discussion, which is I just call “Ask the professor,” And it’s another space for it. I always have to tell them every time I put the same heading and I say, “Don’t ask personal questions here, [LAUGHTER] but if you have a question that everyone can benefit from on the topic this week.” So that’s just an example of how I feel that it just makes it easier for me to focus on what’s in each of those boxes, rather than recreating the format. And I think students really take a lot of comfort in knowing after a couple of week: this is the rhythm, this is what I’m expected to do. And so now they’re focused on different content, but they don’t have to relearn a different set of skills. So going back to your question about the rubric, and is it the same? I do that because then they get used to what skills they’re bringing, and then they can hone that same skill over time, they can go back and look at their rubric grading and see their improvement over time. And I’m also a liberal user of the comment section on rubrics. I know that a lot of people tout rubrics for the time-saving part of it. So oh, you can check these boxes, and it makes it easier. And there’s some of that. But I can’t stop myself from also than opening up that box and just writing a ton of things to them. So I think that, again, it’s one of those things where the rubric itself becomes a really good outline. And then it’s just a matter of what I choose to put into my grading when I do the rubric.

Rebecca: High structure is always helpful for folks, that’s for sure.

Michelle: That’s very reassuring.

Rebecca: I feel like there’s a story behind why infographics as opposed to something else. What led you to infographics as opposed to some other alternative format?

Michelle: So like I said, I’ve been at this 27 years, so maybe I’m just getting old, but I feel myself saying in my head: “Well, kids nowadays don’t have the attention span for writing a 20-page paper.” And that’s not necessarily true. But as I talked about before, maybe they just have to be taken through that process. It can’t be just one big thing. And that was long before the internet, people talking about decreased attention spans. But I do think there’s something to matching up with the way in which they use visuals in their daily life. So they utilize memes, they look at infographics in various formats, I find a lot of them in social media in different areas. So I try to structure discussions in a way that is very much like the way they’re already engaging on social media. So yeah, the story is really how could I find a way that it’s something that their mind is already used to, that they can then apply it in this way. And also the other story, my husband is the County Administrator here in our county. And so I was in his office one day, I was the Assistant County Administrator and I was in his office one day and from the League of Minnesota cities, they had an infographic, a stack of them on the front table about: what does county government do? And that was really the last push I needed because it was so well done. And I thought, “Okay, this is what we need, people need to understand what levels of government do.” People often come to the county with situations that have nothing to do with what counties are actually able to do for them, and so what does your county do? What can it do for you? And I just thought, “How brilliant is that? To put it in that way.” And so I just really was sold on on the idea that infographics can really be fairly deep teaching, if they’re done well. We get so used to the academy and everything has to be these long papers, but rather than bemoaning the fact that the attention span has gone away, I’m just finding a way to adapt to that. So how do students engage with each other? And let’s make that happen.

John: And I would imagine students are more likely in the future to be creating infographics than they are to be writing 20-page papers unless they’re going on to graduate study.

Michelle: Yes, absolutely. That was the other part of it is that this wasn’t really authentic… the 20 page paper… particularly in that intro, liberal education, general education course. That was not authentic to what they were going to be doing. And I felt that in political science, if they were going to be political science majors, they would get that deep research component in other courses and upper-level courses in their capstone. And so I felt that it wasn’t necessary to put them through a 20-page paper. And the reason I keep saying 20 pages is because I realized that the amount of content you need in the country study is pretty lengthy to really understand the country, and so it really was about 20 pages, I just felt that that wasn’t engaging to people, no one was engaging each other with a 20-page paper. And so it had to be something that you could look at more quickly, you have to have the skill of condensing the material, you really had to be really pithy with how you were presenting the material so that someone can get it in a very quick way. And so I feel that allowed them to engage. And then even if you assigned a student to engage with another student’s 20-page paper, pair them off, have them do peer evaluation, they weren’t going to do more than one at that length. Maybe if you cut it down to a 10-page paper, you could have them have a group where they exchange it among three or four people, but that’s still not as big as I’m able to get by having everybody in a class post their infographics so that you can all see huge variety of work in the span of scrolling through normal social media. And of course, I’m asking to do a little bit more, but in some ways, it’s almost that they don’t even notice that part of it because they’re just scrolling through and saying: “oh…” Comments that I get most on the first assignment have to do with “Oh, I’m so glad you chose to do that country, I’ve always wanted to go there. Have you ever been? …because it looks so interesting.” So it doesn’t start out as this deep political discussion. But because of those comments in the first week, I’m convinced of the engagement that starts to come from it. One part that I didn’t mention is I also put in the directions, that if they want to add something, like a fun fact, about the country on any of these infographics, they should, and I’m surprised at how many students then choose to do that on every single one of the infographics at the bottom. There’s a fun fact. And I’ve been amazed by what they’ve come up with. There are things that I don’t know about these countries. I thought, “this is fun.” So I do think that students take to it, they want to show other students their country, they want to say, “Hey, I learned this thing. Isn’t this cool?” And then I get to learn things I didn’t know.

Rebecca: Sounds like a great way to build community and to have a first interaction with a discipline, which sometimes can be off putting. And this sounds like a really positive way to get people interested in a discipline as well.

Michelle: Yeah, I think so, and I hope so. I hope that that does make students more likely to maybe take an upper level, look at us as a minor or major. And I will say, talking so positively about this assignment, because it’s worked so well. But I can also talk to you in a podcast about all the assignments that haven’t worked well, that I don’t get this excited about, that I can’t say were this level of success. And so when we were talking about coming on the podcast, I wanted to highlight this one, because of course, it’s the success. I wanted to show what has actually worked. But, of course, there’s a lot of ones that haven’t. So I do feel that I’ve hit on something here that can be very useful, that I’ve been really satisfied with, and my students seem to be really satisfied with.

John: We always end with a question. What’s next?

Michelle: So that is a great question. I’ve been thinking about that for a while. How do I take this to the next level? So I think one thing that’s next is sharing this out more. We’ve talked a few times about looking at other faculty and seeing if they would adopt something similar to this. So I think that’s one aspect. I think another aspect is to just continue to refine the assignment in a way that gets the most student engagement. I’d also like to see these assignments, if we can have them maybe printed for my asynchronous online classes. This particular class I teach mainly online now, I rarely teach it in the classroom another colleague has those sections. And so, right now, it’s simply archived either as the students own individual file, or in D2 L. And so I’d like to think about ways in which maybe we can archive them, maybe print out them in poster form and have them in the department so that people can maybe again, engage, get interested, a student walking by who’s not in political science who’s hanging out in the hallway waiting for their next class to begin, and starts reading the stuff on the walls and start saying, “Oh, that’s kind of interesting,” and maybe starts to get engaged. So looking at it outside of the particular classroom, I think would be a really exciting way to go with it.

John: Well, thank you. I’ve really enjoyed hearing more about these assignments. And I think it’s something that could work in many disciplines.

Michelle: Yeah, I hope so. And exciting for me would be now to sit back and kind of watch what other people do with it. And oh, okay. I hadn’t thought of that. That’s really cool. That’s always the most exciting part for me from the CETL angle is really just seeing how people take off with something in a way that I wouldn’t expect. And it’s been a pleasure talking with both of you.

Rebecca: I always love a good conversation about interesting assignments. So thanks for chatting with us and sharing in depth how the assignments work over the course of the semester.

Michelle: You’re welcome. My pleasure.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]