343. Writers’ Groups

Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, James Lang, Sarah Rose Cavanagh, and Mike Land join us to discuss their highly productive long-term writing group.

Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College.

Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego.

Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University.

Show Notes

  • Cavanagh, S. R. (2016). The spark of learning: Energizing the college classroom with the science of emotion. West Virginia University Press.
  • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning. John Wiley & Sons.
  • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed. John Wiley & Sons.
  • Jensen, J. (2020). Write no matter what: Advice for academics. University of Chicago Press.
  • Scrivener
  • A General Education – Jim Lang’s substack account
  • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge. Beacon Press.
  • Reading groups at Oswego (in the last three we were joined by Jessamyn Neuhaus and colleagues from SUNY Plattsburgh):
    • Lang, J. M. (2016). Small teaching: Everyday lessons from the science of learning.
    • Lang, J. M. (2020). Distracted: Why students can’t focus and what you can do about it. Hachette UK.
    • Lang, J. M. (2021). Small teaching: Everyday lessons from the science of learning. 2nd ed.
    • Cavanagh, S. R. (2023). Mind over monsters: Supporting youth mental health with compassionate challenge.

Transcript

John: Faculty writing groups can help motivate writing, provide peer feedback, and lead to higher quality writing products. In this episode, we explore a highly productive long-term writing group.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are James Lang, Sarah Rose Cavanagh, and Mike Land. Jim is a Professor of Practice at the Kaneb Center for Teaching Excellence at the University of Notre Dame, the author of 6 superb books on teaching and learning and is the author of a regular column in the Chronicle of Higher Education. He was the founding editor of the West Virginia University Press series on teaching and learning, and is now a co-editor of a new series at Oklahoma University Press. Jim also was the founder and long-time Director of the teaching center at Assumption College. Sarah Rose Cavanagh is a psychologist and the author of four books related to teaching and learning. She is the senior associate director for teaching and learning and associate professor of practice at Simmons University and is also a regular contributor to The Chronicle and many other publications. Jim and Sarah are regular keynote speakers and have both provided keynote addresses at SUNY-Oswego. Mike Land’s early writing and editing experiences included 15 years of newspaper journalism, a masters and doctorate in Creative Writing at the University of Missouri-Columbia, and 23 years of teaching journalistic and creative nonfiction at Assumption, working for many years in the office next door to Jim Lang’s and a short walk from Sarah Cavanagh’s. He’s an Associate Professor of English and Director of the Community Service-Learning Program at Assumption University. Welcome back, Jim and Sarah and welcome, Mike.

Sarah: Thank you.

Jim: Thank you.

Mike: Thank you.

Rebecca: Today’s teas are: …Sarah, are you drinking tea today?

Sarah: Never. You know me. I’m drinking seltzer. It’s too late in the day for coffee. [LAUGHTER]

Rebecca: How about you, Jim?

Jim: I had several cups of tea already today, it’s orange pekoe? And then my last one was English Breakfast actually at about one o’clock.

Rebecca: Nice. How about you Mike?

Mike: I am sampling some Bengal spice which the box tells me is brimming cinnamon, ginger, cardamom, and cloves and it’s an adventurous bland.

Rebecca: Does it taste adventurous?

Mike: It does. Everything about this is an adventure to me. [LAUGHTER]

John: And I have a blueberry green tea today.

Rebecca: And I have not my favorite tea. [LAUGHTER]

John: And what do you mean by that?

Rebecca: It’s not a good tea. It’s the first time I’ve ever had it. It’s supposed to be gingerbread. And I think it’s kind of disgusting. [LAUGHTER]

John: Now you know why we never get any sponsorships from tea companies.

Rebecca: I didn’t say where it came from. [LAUGHTER]

John: That’s true, but I’m sure there’s probably not too many gingerbread teas out there.

Rebecca: There shouldn’t be. [LAUGHTER]

John: In past podcast discussions with Jim and Sarah, we had heard about the writing group that you’ve all participated in. Can you tell us how this group got started?

Sarah: I guess I can take that one. I think it was about 2014…2015. Mike, do you know the year? He’s giving me a big nod. 2014. And Jim had recently invited me to write the Spark of Learning and his first series at WVUP and Mike was working on a book…had just come off sabbatical doing a cross country trip. And Jim was working on the first edition of Small Teaching. Since all three of us were working on book projects, Jim said, “Hey, why don’t we form this writers’ group.” And we started meeting. And it was really such a beautiful process to write my first book with my editor [LAUGHTER] reading every chapter as it came out, was a very developmental process. And as well as forming some really great friendships.

Rebecca: Sounds like a great start to a long endeavor. Can you talk a little bit about how regularly this group meets and if you still continue to meet?

Jim: We do continue to meet, but not regularly, as you might expect, [LAUGHTER] especially just because we were initially together at Assumption and were all working in the same area, and I just mean sort of geographical, and now Sarah’s in Boston, I’m going back and forth between Worcester and South Bend. So it’s a little bit harder to get us together in the same place at the same time. So we kind of meet on a more ad hoc basis at this point. if one of us has a project that we want feedback on will sort of send an email around saying “I need help, [LAUGHTER] and so let’s get together and can we have a writers group coming up?” So we’re still probably doing that maybe six to eight times a year, I would say. Maybe we were trying to do it every month but now it’s kind of morphed into that kind of more occasional kind of schedule.

Sarah: So we usually squeeze in a Christmas one.

John: That’s true.”

Sarah: So usually a mid December writers’ group. Some of them are in Zoom just because of how we’re spread around but the Christmas one is always in person. It’s really nice.

Jim: Yes, we have holiday themed groups. [LAUGHTER]

Mike: …with teas.

Sarah: Well, for some of us.

Mike: …or beers, but yeah, we’ve tried to have a couple of year that are also just social and in person. We’ve got more used to Zoom,as everyone has, and since we’re in three different…well, Jim and I live two doors from each other, but it’s harder to get us all together physically in the same space, but we’ve try to do that two or three times a year.

John:. When you were meeting regularly at Assumption University, what happened during the meetings when you got together? How long were the meetings? And what did you do in the meetings?

Jim: That would also depend upon where everyone was, in terms of their writing projects. A typical meeting would last maybe 90 minutes. Mike’s especially a social guy, [LAUGHTER] he would sort of want us to start with some social time, we sort of built that in. And so we’d have a little social time initially, you might be around a meal, for example, for the first 15 to 30 minutes, just catching up on our lives. And then we kind of just go through the project, essentially. So we essentially will just pick one off the pile, and say, “Okay, so here’s my essay this time,” like, for example, I might be doing a Chronicle essay or a chapter of a book. You have to read it in advance. So that’s an important thing to note, that we don’t read out loud, which sometimes writers’ groups will do that. You read in advance, we send the stuff out at least 48 hours in advance. You’re expected to read that material, come into the group having already written comments, either online or actually some of us still use the paper versions and bring those in as well. So then, essentially, the person whose material that we’re critiquing is supposed to just sit and ask questions, and the people that have read will kind of go through, essentially page by page, there might be like an initial comment, “overall, this is what I think of the piece,” but then we kind of go through page by page. That doesn’t mean every page is commented on, because it depends on where the issues are. But say, each person’s piece could take 15 or 20 minutes or so to walk through it. And then we move to the next one. And so we move around the circle that way. That’s the basic process, but what am I leaving out?

Sarah: No, I think that’s most of it… Kind of the flow.

Rebecca: Can you talk a little bit about the benefits of having formed the writing group and what other people might think about in forming a group?

Sarah: It is so invaluable, I think, in so many different ways to have first readers who can read something. It’s so vulnerable writing and putting your thoughts out into the world, and to have a safe place where people can give you feedback on very early drafts, I think is so valuable. And it’s also a really good test of are you expressing your ideas well, and are your references sound. I’m always telling Jim, people who don’t know sports aren’t going to understand this reference, and he does the same for my sci-fi references, and so that kind of tells you what a broader audience might think of your ideas. And it’s just a place to bounce your ideas around as well, like sometimes we do a lot of line-by-line feedback, and Mike is awesome at that from his journalism background. But we also sometimes just ask advice and just say, Is this even working? Should I draft this project and go to something else? Kind of really broad ideas as well. How about you, Mike, what do you see as the benefits?

Mike: The things you said. I think it’s a lonely pursuit, and if you don’t have an audience of people who are intelligent readers to let you know whether you’re totally going in the wrong direction or not, you’ve been staring at what you’ve been working on for so long, you convince yourself that yeah, this makes perfect sense, or this was a good move, or they’ll laugh at this, whatever it is. And so at some point, you got to try it out on audience members you trust and who also understand the writing process, because they themselves are writing and have written a lot. So they differentiate some things, maybe what not to do or to do. There’s a temptation in fiction writing, when I was in the fiction writing program, we were warned about trying not to turn the short story of the person being critiqued just into the story you had felt like writing instead. [LAUGHTER] Well, I figured out what the person is trying to do, and help them do that, and stay locked in on that, to some extent. So yeah, there was a lot there. And it is an extra deadline and something to get you moving.

Jim: That’s an important thing to know, too, that it was oftentimes a way of saying, “Okay, I have this book coming out, so I want to make sure I go chapter by chapter, and so I would make the writing group deadlines, my internal deadlines for getting those chapters done.” That’s a really, really helpful structural enabler, when you’re trying to especially do like a long project. The other thing I mean has come up in actually both these comments from Mike and Sarah, about the audience. And that’s true in terms of your audience. Think about your ideal audience who you’re trying to really write for. It’s also really true for your editors and potential publishers. And we often talk about that, like, “Okay, who would really be the right place to submit this to?” And we might be able to share examples of the publishers that we’ve worked with or strategies for reaching out to an editor, maybe we have a connection, and we know somebody. So we can have also that as well. So especially if you have books in your writers group who have published other things, you can start building networks around a writer’s group and I think that’s really important, too. Again, I think what Mike and Sarah are saying about audience is a real big benefit of you’re sort of thinking about the readers that you’re trying to reach, and you can often refine those. The example that Sarah gave about references, like if I’m writing to a community of people who love science fiction, I don’t have to worry about those references. But if I am going to expand beyond that audience then I have to either explain them or make more universal ones, and I think that process of like hearing at least two other readers, helps you expand a little bit in the number of people that you can reach with your writing.

John: And I would think having that relatively timely feedback would be much better than writing a big chunk of a book, sending it off, having it go out to reviewers and then getting reviews back when it’s perhaps a little bit late to make some of those major changes, that getting that more immediate feedback, I would think, would be helpful in keeping the writing project going in the direction you intended.

Jim: Absolutely. For example, I had like a little mini readers group with my Chronicle editor. Now I’d submit my work to her, and she would really sort of hack it to bits, [LAUGHTER] in a nice way, she’s a great person. But over time, I’ve sort of recognized, this is what she wants, this is what she looks for, this is what she changes. And I kind of learned to sort of mold myself into that kind of form that she wants. And the same thing is true for your writers’ group, you can sort of start to anticipate the things that they’re going to point out to you, which are often little habits that we have, which are not always great ones. [LAUGHTER] I always remember that Sarah would always point out that I used the word “so” all the time as like, “and so this,” and I never would have noticed that unless she started to circle all my “so’s” and then “look how often you do this,” that was like a really helpful thing for me to know. So now I’m aware of it, I still use them, but I’m just more aware of it now. So it’s very helpful to see other people’s perspectives and then you can start to anticipate that.Sarah sometimes uses phrase, “I had Jim in my head when I was reading this sentence,” or “I had Mike in my head…

Sarah: Yep. [LAUGHTER]

Jim: …telling this anecdote” or something like that. So that’s a really helpful thing to have as well.

Mike: Yeah. And it’s interesting, you’ll learn those things about each other, and then we can decide when to accept or not accept that, right? [LAUGHTER] In my case, and I think it’s true of Sarah, we can anticipate that if we throw in one too many anecdotes, we know Jim’s going to be there waiting to pounce on our extraneous anecdote that we love. And I’m somewhere in the middle, from the South, so somewhat more elliptical in my storytelling and less linear. So I’m favor of the extra anecdote. But there are times when you can tell, when you looked at it, and you have that voice and you go, “Okay, this really is extraneous, get it out of the way, don’t even bother other people with it.” And that’s that kind of thing and that’s kind of why we assign undergraduate students to do peer editing of each other is to learn to be editors, and to hear those voices too. So it’s really useful that way.

Jim: Mike, you know about other readiness groups and this question, John asked us about, like, guidance for other groups, can you give some examples, Mike, if you know of other writers’ groups that have sort of different ways of working?

Mike: There have been some groups that seem to exist just for people to pat each other on the back. And they don’t really do much criticism at all. A friend of mine who’s dyslexic said that he was in a group where you could never correct anything ever, grammatically or stylistically, which would drive me crazy, because I would want people to note every time I slipped up on something. So those are a couple of examples, and I’ve been trying to come up with some more, but it seems like there are a lot of variations in the rules in how they do it.

Jim: Sarah, you have also one that’s more like keeping you accountable, right?

Sarah: So I have another writers’ group with some friends at Tufts University, and we call it “writing and hijinks.” And so we gather together in a coffee shop or somewhere and we just write quietly, although one of them, like Mike, is very social and has trouble with the quiet part. [LAUGHTER] And so we just write, it’s just dedicated writing time for a couple hours, and then we go out and have fun. Although we’ve been sliding more and more towards just the fun and not doing the writing [LAUGHTER] over the last few years. This is motivating me to say to them, maybe we should bring back the writing part.

Rebecca: We have a writing group at Oswego that functions like yours, Sarah, that you’re talking about, where it’s just kind of some dedicated time, accountable time, this is the time we’ve set aside to write…

Sarah: Right.

Rebecca: …which can be really helpful if you have a designated time and space to do that, if you aren’t good at scheduling it on your calendar otherwise. Julie

Jim: Joli Jensen has a book about writing from the University of Chicago Press, and she actually makes the argument that writing groups for academics should not be content critique groups, which is what we have, actually. And she makes the argument that it’s not helpful, for example, English professors to be working with a psychologist because they won’t get your references and all that kind of stuff, and it won’t be very helpful to you. But actually I would argue that what we’re doing here, because most of us are running for, like broader audiences, the content critique part is really important. If Mike and Sarah don’t understand what I’m saying, well, then I’m not gonna be able to reach a broader audience instead of just writing specific to my little expertise audience. So that’s another way to think about distinguishing content, whether you focus on the content or just the writing, the sort of grammar and the stylistic part of it. So that’s another way to think about what are you actually trying to do in your writers’ group? Are you trying to expand beyond your discipline and getting different kinds of feedback? Or are you just trying to focus on, as Mike said, maybe patting each other on the back or accountability or style and grammar? So there’s a lot of different ways you can go.

John: I think that would depend a lot on who your intended audience is. So that if you’re all psychologists, or all economists, or all mathematicians, if you’re going to be submitting things to journals, it might not be a bad audience. But for a general audience, having someone who is not necessarily an expert in the field, seeing it from a different perspective would lead to a lot of really productive comments in terms of what some of the hidden assumptions are, the implicit assumptions in the writing.

Jim: Absolutely, you can actually even think about it as a teaching thing too, I mean teaching to like a gen ed class, their first-year students, that’s almost like a content critique group in the sense you can’t take too much granted about background knowledge in discipline, like a really focused content critique group, all from a discipline, you’re essentially teaching a graduate seminar. And so you’re looking for different things from those experiences, and you’re being required to explain different levels of things to those two audiences.

Rebecca: So I’m curious what participating in this particular writing group with the three of you has done for you as writers? How has it shaped your writing? And why are the three of you a good group for each other?

Sarah: I think that one of the things that has really shaped my writing process, this is beyond all of the wonderful things that we have said already about audience and things like that, is being in a writers’ group with two English professors is really valuable. I tested out of intro writing in college, so I actually never took a writing course of any sort, and then decided I wanted to be a writer. And my writing is very organic and intuitive. And working with these two really helped me like the things that they would correct. Like I didn’t know periods went inside the quotes, until Mike corrected me. [LAUGHTER] But the first year, we were writing together, and you would think I would just absorb that from reading, but I just never noticed that. And so really tiny things like that, but then also just thinking about the structure of your writing and the organization of your writing, because I had never really done that for this kind of general audience writing. And so having this writers’ group with writing professors is really amazing, as Jim would sometimes send me to the whiteboard, when we would meet [LAUGHTER] at Assumption, because it’d be like, you’d have a lot of really interesting ideas here in the writers’ group, but they’re just in a big pile,[LAUGHTER] and he’d have me map out, do a literal map of my work. And that was very, very helpful.

Mike: That’s one of the fascinating things about being in a group is working with two people that have such different composing processes. And Jim, you’re fairly linear…

Sarah: fairly…

Mike: kind of have it in order [LAUGHTER] and smoothly.

Sarah: …the computer. [LAUGHTER]

Mike: So what was the computer program that you’re using that divided the screen into quadrants?

Sarah: Yes, Scrivener, you can put your bits of writing on literal digital index cards, and just move them all around.

Mike: Yeah, which is something I’ve seen poets do. I’ve seen poets tear their poems apart, line by line and move the lines around in the coffee shops. So it’s just two equally effective ways of doing that. And I guess I’m somewhat in the middle, but with the road trip narratives, obviously, there was someone’s this town and that town, but it’s fascinating to watch how people’s brains work differently.

Jim: Yeah, I think for me, Sarah and Mike are different kinds of writers. But as a result, I’m kind of benefiting from both the sort of storytelling advice that comes from Mike and from Sarah, again, more of the research, making sure that I’m supporting my points well, and thinking about other perspectives on the topics I’m addressing. And it’s helpful to get both those things, because I’m getting different things from others, like editors, for example. I like the idea of starting a piece of writing with a story or something like that, my editor at The Chronicle always wants me to lead with the thesis, essentially. And so I’m trying to put some of Mike’s ideas advices into that kind of storytelling opening, but also trying to satisfy my editor. So anyways, it kind of helps you develop new strategies and see how strategies might land with different people and Sarah’s especially good at ways that I should include more readers, more thinkers from different backgrounds, and essentially making my writing more inclusive. And I think that’s really valuable as well, and making sure that I engage with the research that I might not be aware of, in the teaching and learning fields. So that’s really important, too. So I can look at my own writing and see how it’s benefited from both their perspectives.

Rebecca: I think one thing that I’ve benefited from when I’ve participated in writing group opportunities is actually seeing early stages of other people’s writing, and just what that looks like and feels like and sometimes provides strategies to future me [LAUGHTER] when I’m stuck, [LAUGHTER] like, “oh, wait, this other person would have attempted it like this.” And when I felt stuck on something, sometimes I use the strategies that I’ve observed other people using, which can be really beneficial, that things that I wasn’t taught or hadn’t been exposed to otherwise.

Sarah: That’s great.

John: You mentioned future me. And one of the issues that we know from behavioral economics and psychology is that when given a choice between pursuing your long range objectives and immediate gratification, immediate gratification sometimes wins out, [LAUGHTER] because it’s always easier to start these things tomorrow or next week. And one of the things you mentioned earlier, Jim, was that having the regular meetings of the writing group provided a bit of a commitment device that you want to get to the meetings with some new material you want to share that material before it. Do you think that has helped increase the pace in which your work has taken place?

Jim: For me absolutely, yeah, having that structure there. I think structure is always a really helpful thing in teaching and writing. If you sign a book contract, the deadline might be two years away. So you know that two years, an editor might check in after a year, but like, you need to have more structure in that to kind of get yourself moving along. And I’m a pretty structured person, but I can also fritter time away. I’m pretty good at that, too. [LAUGHTER] But I’m self aware enough to know I need a little help in terms of structure. And so it definitely has pushed me to become more productive.

Sarah: Absolutely. I agree. This whole conversation is making me want us to do this more regularly, [LAUGHTER] instead of the as needed, because I think about especially the beginning of the writers group. And when I was writing Spark of Learning, my daughter was a toddler. And she was only in daycare three days a week, and I was on the tenure track and just doing a billion different things. And I do not think I would have written that book if it weren’t for writers’ group monthly meetings. I think at that point, we were really regularly meeting in person monthly. And just finding the time at four in the morning [LAUGHTER] or terrible times often, but I don’t know that I would have gotten it done if it were just like, “oh, I have a contract and I must do this giant thing.”

Mike: That definitely helped this begin…the structure. I don’t have a long-term project I’m working on currently so hard for me sometimes to come back around and really have anything that’s worth them looking at. But yeah, you mentioned those long period of the book contract, and there’s a long period of only doing shorter things, whereas so there’s not that continuity from chapter one to chapter two to chapter three, and you’re eager to write those next two pages that are the bridge to the next thing, so group can help with that as well. It seems like a lot, but what happens in writing group is when someone is working on a Chronicle column that needs to be turned in in 48 hours or letters and things like that. And I was wondering if you can give examples of that kind of stuff when the group comes into play. That’s not the big primary thing we’re working on, but it’s kind of the emergency stuff.

Jim: We’ve done a little bit of that, but it’s kind of hard because especially for an upcoming deadline, I think typically we’ve handled that when someone just needed something very quickly, we just handled that by email. But the writers’ group is available in those emergency situations, because we already have a quick sense of like, if Sarah sends me something, and it was for a quick turnaround, it’s for The Chronicle, I can be able to spot very quickly a couple of things that you could maybe do in that short window. And likewise for me. So it’s challenging, though, because you want to have time for people to have opportunity to read and think about that as well. But it’s available for those situations. So it’s good for that.

Rebecca: I can imagine that you’re strong relationship with one another, having written with each other for so long allows for that quick turnaround by email, because you can sense the tone someone’s providing some feedback in [LAUGHTER] and it…

Sarah: Help!

Rebecca: …has a different kind of context. [LAUGHTER] Yeah.

Jim: Absolutely.

Sarah: And it’s such a pleasure, like it’s hard to ask for emergency work from people, even people you’re close with, and maybe even especially people you’re close with, because you know, they have so much else going on. But it’s such a pleasure to read these gentlemen’s words, that it’s not a burden at all, even the emergency ones because it’s like, “Oh, I get to relax, not with a cup of tea, [LAUGHTER] and read some words from Jim or Mike.

Jim: That’s true. That part of it is good, because you know that you can help them in the emergency because they’ve helped you in the past, that same situation. Right now, you know, I’m sort of developing a substack account. And I haven’t really used the writers’ group for that. And as a result, I need to have my wife proofread for me,[LAUGHTER] because substack doesn’t have an editor, and I’m really annoying her by this process. So I need to start using my substack columns for writers group stuff as well.

Mike: Yeah, it is really helpful and we know that there’s just a lot less explaining to do amongst the three of us. If I were going to send something cold to another friend, I think I would have to explain a lot more about what was going on for them to get it. So that’s incredibly useful that way. But it’s funny how different it is from like growing up in a newspaper family, and I would interview someone in the morning and I would revise it four or five times in a period of like four hours and hand it to my editor who would then tell me to change three things. And the whole thing from beginning to end was less than a day. But there was no deep thought, there was no deep reflection or big structural things, and then my time sort of expanded magically as I became an academic and suddenly you can get trapped in a lot of whirlpools of uncertainty about which way to go or that way to go. Whereas in journalism, there wasn’t much choice a lot of the time.

Sarah: Writers note, I love that “whirlpools of uncertainty.” If I were in writers’ group, I would say underline it, put a little exclamation point next to it. [LAUGHTER]

Mike: Well, I was a really terrible student in an undergraduate poetry workshop at the same time my sports editor was trying to teach me to write in a linear way. And my teacher said at one point, Mike what you’re writing, as poetry is prose with line breaks in random places. And he said, “Let the language go where it wants to go.” And I was having a hard time imagining telling my sports editor that on [LAUGHTER] Friday night during high school football season, but it seriously took a long time for me to learn I could do both. I could loosen up and be this one kind of writer, and then be this very mechanical quick writer. And that took years as an undergraduate to figure out how can I do both of these things at the same time.

Rebecca: One thing I heard Sarah say is that currently you’re meeting as needed, but then I also heard in the conversation, that maybe we need these things, and we just don’t know it. So having it regularly on the calendar could be helpful. [LAUGHTER]

Sarah: Mm hmm.

Jim: Yes, we’re convinced now this conversation is very helpful to us. [LAUGHTER]

Mike: Yeah, and plus this almost the end of the semester, so suddenly, we’re gonna have a lot of time on our calendars.

Sarah: …well, those of us still teaching. [LAUGHTER]

Mike: Yeah.

Sarah: I’m at 12-month contract now.

MIKEL Oh, you are?

Sarah: Yeah.

Mike: For you, year round.

Rebecca: Me too, Sarah. [LAUGHTER] Maybe we need a separate writers’ group. [LAUGHTER]

Sarah: Or therapy. [LAUGHTER] …support group.

John: What sort of advice would you give to other people who are thinking about forming a writers’ group?

Jim: Actually, there have been been other people in the writers’ group over the years, I’ve been in other writers groups too. For me, the thing that I look for is people who I thought were sort of ambitious and were writing, because you don’t want to be the person who is trying to gather people together, when they’re just not writing, they don’t want to write, you don’t want to be like the taskmaster in a writers’ group. So find people that are writing and are interested in writing, and find people that you find interesting, you want to hear more of their thoughts. So that has always been a driver for me. And sometimes, if I’m looking for people that I want to hear their thoughts about my own writing, and I want to talk to them about their own writing, I’m gonna be more interested in going to that writers’ group and participating. And so those are sort of two general points I would make: look for people that are writing and interested in writing, and then people who are interesting, you want to see it develop and getting access to their ideas as they’re developing. And I would just say, most people, I think, are flattered, you would like to hear and read their work. So if you identify someone that and you might say, “Well, oh, you know, they’re probably too busy, or they probably already have other people to work with,” that might not be true. And so extend the invitation and see if they want to join you in pushing writing forward.

Sarah: And I think I would add, it’d be hard to figure this out, [LAUGHTER] but one thing that I appreciate about this group, and I would want in any writers’ group is having a similar level of like sensitivity and openness to feedback and criticism. So I think that all of us are really great at taking feedback and taking constructive criticism, without either taking it personally, or I think Mike referenced this, but sometimes the Jim in my head or the Jim on my page, I’m like, “Yeah, I’m not listening to you.” [LAUGHTER] But then a lot of times you do. And so t o people who are at that level of like, open to hearing criticism, but also confident enough to stick with their gut if they feel it’s right. And yeah, and so that’s a delicate thing. And it’s hard to figure out about someone without actually engaging in writers’ group. But I think if they were in your department meetings or on committees with you, kind of keep an eye and get a sense for how open they are to other people’s thoughts and how willing they are to give them.

Mike: Yeah, and along the line with open is something I always warn undergraduates about, but in grad school it would come up with separating the characters on the page, even the narrator, that that person I’m writing the first person about, it’s Mike, the narrator on that page, but it’s not necessarily all of what I am. So, ‘cause when you’re talking to someone, then you can point out stuff that you think is maybe sexist, or is whatever, but be able to talk about it in a separation where you’re not personally attacking the writer, but that you are holding that in a certain place. I think we’re really good about those sorts of things, we know the level each of us are trying to get to, we’re just trying to help each other get there and again, smooth out anything that the audience might misinterpret. That’s a mental discipline that takes a while to be learned. And I’ve had friends who have been in groups where things have devolved to name calling, and to really unfortunate situations where the writing group had to be broken up, and so I’ve even heard that happen in academic settings, like in actual seminars. Fortunately, I was not part of any of those, but where people went stomping out of the room. And so yeah, having the maturity in just the way you carry yourself in life, I guess, to know how to treat other people and work with them constructively, a really important thing. But otherwise, in terms of advice, let’s say that whatever kind of writing group you want to have, you want to sit and just right beside each other and then go have drinks that what this one’s gonna be, are you all aiming to publish and you’re trying to help each other get published? That can be a very different mentality, depending on what you’re trying to do. That’s my two cents.

John: Do any of you have any other thoughts about writing groups that you’d like to share with our listeners?

Jim: I will say just as an editor, you know, editing the book series, I can also sometimes tell people who had writers’ groups when they turn their manuscripts in who had prior readers. So just from my perspective as an editor, I absolutely recommend this, forming a writers’ group, especially if you’ve signed a contract, and you’re intimidated by it. The fact that you have this deadline, you have this big project coming up, A writers’ group can not only put some structure on to the process but also forestall a lot of the work that I will have to do as an editor. So you can make my life easier when you’ve had some prior readers who kind of work with you to iron out a lot of the kinks that have developed, as any writing project will have in the drafting.

Sarah: Yeah, my editor was very grateful for writers’ group. We met monthly, my editor for Mind Over Monsters and she was wonderful. But writers’ group was like a third character in the room. [LAUGHTER] Because we would meet monthly, I’d be like, “Alright, well, here’s the chapter, but this is what writers’ group says, and they think that I should reframe this.” So it was really like the three of us, or I guess, the four of us, melding you two into one being in the conversation and in the room. And that was an interesting process.

Mike: I think another thing that’s been interesting to me is that because Jim and Sarah have books out a lot of the time, and there’s a good bit of talk about the publishing aspect of it, there are times when say, Jim might know someone that one of us should hit up, or contact, or things like that, or how to interpret an editor’s reaction. And there’s been a good bit of conversation about those things where I kind of felt like I know, these editors I’ve never met. [LAUGHTER] I’m sure that’s a big stopping point for a lot of people is they’ve written this thing, and now they don’t know what to do. They don’t know what the path is, and how to get there, how to get someone’s attention. And so there’s a lot of talking back and forth about things like that.

Rebecca: Do you have wishes for your group moving forward?

Jim: I guess, more regular meetings.

Mike: …more regular meetings, yes. [LAUGHTER]

Sarah: Sounds like a way to save Jim’s marriage.[LAUGHTER]

Mike: I guess, I should sell them another road trip so I could write another road trip manuscript.

Jim: Yeah.

Sarah: There you go. I would read that.

Mike: As we sneak toward possible retirement…a travel book…that could be the plan.

Rebecca: So we always wrap up by asking: “what’s next?”

Sarah: Well, more regular writers’ groups, I guess. But I think as Mike just referenced, he’s retiring. So he’s going to have a lot more time to write coming up. And Jim and I are both, I think at the phase of thinking about next book projects. And so maybe it’s going to be a new day for writers’ group.

Jim: That’s true. I’m actually doing the final, final, final revisions of my current book manuscript, which are due on April 22nd. So that will be coming up shortly. And so I’m in that phase where like, I’m kind of finishing everything up, and also the editing and all that stuff, and copy editing, but still at least starting to open up in the sort of creative part of your brain about like, what might be next? And so that’s maybe one final thing I might say about writers’ group is sometimes we will come in with just like a one pager of ideas. And Sarah has done that several times. And we kind of give her feedback like, this is the one that seems like it might be the best path to pursue, for example. Anyways, I’m in that space right now. So I hope that for me going forward writers’ group might help me land on the right project next.

Mike: I should mention too that just the venting of frustration is good. And it just crossed my mind, one of Jim’s funnier Facebook posts of all time was when you were finishing up a book and what you posted about how you felt about the book. I don’t know if you recall this or not?

Jim: Yeah, absolutely.

Mike: So tell them. [LAUGHTER]

Jim: So I just remember talking about the fact that whenever you finish a book, I’m just so sick of it. At that point, I never want to think about it again. [LAUGHTER] And then of course, a year later it comes out and you have to think about it all the time. [LAUGHTER] I’m a person that has multiple interests, and they sort of change on a regular basis and like, by the time I finished a book, I’m like, “Okay, I’m done with this now, what’s the next thing?” But then, you know, hopefully a year goes by when it’s coming out, and then you get interested in it again during that time, hopefully.

Mike: Well, the specific line I remember was, “I hate, hate, hate, hate, hate you, stupid book.” [LAUGHTER]. It was that quote.

Jim: Yeah.

John: Well, I can see it’s probably good that you focus on the writing and not the marketing of these books. [LAUGHTER]

Jim: Yeah.

Mike: It wasn’t this book, it was a different book.

Rebecca: Well, thank you so much for joining us. It’s always a pleasure to talk to you about your work. And we’ve been hearing about this writers’ group [LAUGHTER] for a while. So we were glad to have the opportunity to dive in a little more deeply.

John: And we’re also grateful that we’ve been able to benefit from the writers’ group. I was just thinking back, four of our reading groups over the last few years were products of this writing group.

Jim: Wow.

John: So we appreciate the work that you’ve been doing.

Jim: That’s very cool.

Sarah: Thank you. It’s an honor.

Mike: Yeah, thank you.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

342. Infographics

When papers and projects are due at the end of the term, students often procrastinate even when the projects are carefully scaffolded. In this episode, Michelle Kukoleca Hammes joins us to discuss how a series of infographic assignments, combined with peer and instructor feedback, provide an engaging and productive learning experience. Michelle is an associate professor of political science and a CETL Fellow for the Center for Excellence in Teaching and Learning at St. Cloud State University.

Show Notes

Transcript

John: When papers and projects are due at the end of the term, students often procrastinate even when the projects are carefully scaffolded. In this episode, we discuss how a series of infographic assignments combined with peer and instructor feedback provide an engaging and productive learning experience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Michelle Kukoleca Hammes. Michelle is an associate professor of political science and a CETL Fellow for the Center for Excellence in Teaching and Learning at St. Cloud State University. Welcome, Michelle.

Michelle: Hi. Nice to be here.

John: We’re looking forward to talking to you. Today’s teas are:… Michelle, are you drinking tea?

Michelle: I am. I don’t always drink tea, but I made sure this morning I did. And so I’m actually just drinking a green tea, a milk tea. It’s not quite as good as others I’ve had. I have a colleague from Wales who makes the best green tea, but it’s good this morning.

John: …and Rebecca?

Rebecca: I have a Hunan Noir Tea, which is very tasty.

John: You left out the Pinot.

Rebecca: I know. [LAUGHTER]

John: Okay. And my tea is a black raspberry green tea.

Michelle: Sounds nice.

Rebecca: So we’ve invited you here today, Michelle, to discuss how you’ve been using infographic assignments in your online comparative politics classes. When you first started teaching this course, what kinds of assignments did you use?

Michelle: I use a variety of assignments. I’ve been teaching this course for 27 years now. And then this is mainly for my intro level comparative course. But I’ve adapted it for upper levels too, so mainly traditional assays early on. As I moved to the online environment, I would still have them do a major paper of a country study. And so pretty traditional assignments for the most part, but I wanted to make some changes for the online environment especially.

John: So what made you decide to change to infographics?

Michelle: Well, a few things. One is I felt that students were doing these papers that were quite lengthy in their country studies, and they would turn them in at the end of the semester. And we never really had a chance to talk about the elements of them in depth prior to that. And so I really wanted to change that. So it wasn’t just the end of the semester. And they weren’t sharing them with each other, because of the format of them as just being a lengthy paper, it wasn’t the kind of thing that they would be interested in exchanging with each other. And then frankly, I was getting a little bored, [LAUGHTER] the format, not so much the content or their insights, but the format just became very boring. We all know that when you’re working with one individual paper, it can be really exciting to have that dialogue with the student. But when you have a stack of papers, it suddenly becomes too daunting. And so I was just looking for a way to make us all happier, actually.

Rebecca: That sounds like a good motivation. [LAUGHTER] Can you talk a little bit about what the infographic assignment is?

Michelle: So I will talk about in terms of that intro level comparative politics class. One of the things that there’s not time to do in any semester is to really cover every country in the world, obviously. First of all, I wanted to make sure that students could pick their own country that they have an interest in, and that would broaden the number of countries we would cover. And then I switched it to infographic for the online component because there were a few things I wanted to do with it. One is to retain the goals I had in mind for them learning about a country. But also then I wanted to tie in some best practices for online presence in the classroom, for example. My discussions, I was also unsatisfied at the beginning, we learn as we go along, and my discussions tended to also be maybe a little stale, or I’d throw out a question and I would get very similar answers from everybody. They were going back to the textbook and giving me something rather than being creative or energetic in their own answers. And so the way the infographic works is I have about 10 through the semester that they have to do. And so it’s basically breaking up the country study into smaller pieces, so that they have different topics as we go along the different topics in the class. And then before they submit them to me, they submit them into the discussion for the appropriate week. And that way, all the other students can see them, it gives them a lot of different skills. One is that they are able to critique other people’s work, they are gaining more knowledge about a variety of countries. It allows them to practice skills on their own in terms of working together. So I don’t see these always as only individual assignments, although that’s how they’re graded, but kind of a group work assignment that they can all enhance each other. And so there were a lot of ways in which I wanted to bring different elements into the same assignment, particularly because I didn’t want to have to think of how to bring these elements in and do different assignments for each thing I wanted to bring into the classroom. Having one assignment that was consistent really helped them.

Rebecca: It sounds like by having a consistent assignment, you do have to learn how to do the assignment over and over again, that part of it is covered. You do it one time and then it’s clear how to do the assignment and then you can focus on the content and the material.

Michelle: Yeah, exactly. And one of the other things is, unlike just having one country study at the end, there was a lot more formative assessment that could go into it. So exactly that, where we’re able to help build skills up early on. Some of the content early on is pretty simple. The first one is making an infographic about the demographics of the country. And so it’s pretty simple, but it allows them to get the first skills if they’ve never used, for example, Canva, or other programs that allow them to make an infographic. So they get their feet wet with that. It allows them to think about how to communicate in a different way other than a paper, because I also began to think about my intro level students who many would become political science majors, but many wouldn’t. And so in a course that they were using for general education, they’re not going to be writing political science research papers. And so allowing them in different ways to express themselves I felt was another skill that they could bring.

Rebecca: It was nice to see this scaffolding, separating out some of the technical skills from the content, because often we’re doing all of those things at once. And it sometimes can be unclear whether or not it’s a technical skill that is lacking or the lack of understanding of something. So separating those out of it really does seem like it would help have a clearer sense of where students are. And, as we know, [LAUGHTER] if you try to ask students to do too many things in one bigger assignment, then certain things end up edited out. And that might be the thing that was really critical. So it’s nice to have those separate assignments to focus in on those.

Michelle: Yeah, exactly. And I really enjoy the fact that, rather than focusing on grading a long paper, I’m able to go through that process with them. And then also in discussions, it gives me so many opportunities to re-teach something. So if I’m noticing in someone’s infographic that they’ve made an error, or they have a misunderstanding of a concept we’re working with, then all the students in the class can see me correcting that and doing it in a gentle way, that seems a little bit less high stakes for students that because it is just this one assignment, and I’m making this one little correction here, but I think it benefits everybody.

John: So for those students who haven’t done much work with infographics or with graphics in general, do you provide any instructions or recommendations to them? You mentioned Canva, do most students use that? Or is it a mix of tools?

Michelle: They usually do end up using Canva, because that is one of the tools that I put in the directions. And so I have quite lengthy directions in terms of guiding them through accessing technology, allowing them to know that they can use various different technologies, depending on their accessibility to them. There are some really great programs that of course, cost money or pro versions. But I let them know that for our purposes, they can certainly just download a free version of a program. They could really even do it in the Microsoft Office Suite in a variety of ways if that’s where their skills lie. And of course, that’s something that all the students have access to on campus. And so I do get them lengthy directions, they’re not all my directions. I don’t feel like I need to reinvent the wheel. So I just include several videos from YouTube that talk about how to use Canva. And those are sufficient and actually probably better than I can do. And so I put those out there with other directions, I give them alternatives. And then of course, if they’re still stuck, then they can always work through it with me. We can have a Zoom meeting where they pull up their materials, and we work on putting it together.

Rebecca: You mentioned that students were sharing infographics with each other, it sounded like maybe through a learning management system or something. Can you talk a little bit more about what that sharing looks like and the kind of feedback that students are giving one another.

Michelle: So the learning management system we use here is D2L Brightspace, and there is of course, like in most learning management systems, there’s that built in discussion feature, so I just use what’s already there. I don’t mind using third-party applications., but it just tends to make things trickier. So I just use the discussion. Have them post it right to the discussion, and have them use the discussion tools, whether it be just typing in or using some audio or other tools to give feedback to each other. And I find that the discussions are so much more lively. Even more than I expected. The first time I used it, it was for a summer class and I thought “okay, I’m gonna try this now because it’s summer, and no one wants to be sitting writing a 20-page paper in summer. What am I going to do?” So I put it out there for the summer class and I thought, Okay, well, maybe they’re busy with other things or they’ve got excitement in their own life over the summer or they’re working a lot of hours. I had never had such lengthy discussions go on in any class before that. They really enjoyed looking at each other’s work rather than just from me. Many of the students chose countries that reflect an interest they have, maybe it’s somewhere that they’re actually from, although I usually try to encourage them to broaden a little bit. But that also brings such a benefit to the rest of the students. So we do have students who do that. I have students who maybe have gone on vacation somewhere or have a family connection somewhere, I’ve had some that have been stationed in the military, and come back, and they want to talk about what they noticed when they were there, so they put a lot of that in the infographics. I found just use the basic tool, but allow it to be opened up for them. And I also try in discussion, and I do this even in other discussions that I have online, because I usually try to hang back for the first bit, quite a bit. I allow that to be organic. And then I’ll go in and make sometimes just a summary of everything that’s been said or things that I need to correct. So that it’s not simply another lesson from me. But I have that ability to correct when I need to.

John: You mentioned the first infographic assignment had them work with demographic data and display that, to what extent are these assignments tied to the specific course work that you’re doing during each of the 10 assignments?

Michelle: For most of those 10, they’re directly tied to the material for the week. So for example, the textbook will have a chapter where we’re talking about government structures, presidential systems, parliamentary systems, et cetera. And so the infographic for that week will be directly related. So they’re going to do an infographic laying out the government structure. And then another module will be on political parties and how they work in various parts of the world. And so it’ll be directly tied to: “So show me your country, and show me the main political parties in your country. Tell me what their main ideology is. Tell me who’s in power now… which party? And what does that mean for the policies that get put forward?” Another one would be one on current events, the last one that they do is: “Well, what’s going on there now? …and “With everything you know about your country now, what do you predict for the future for them? How do you think they’re gonna solve this?” So a little bit more policy oriented… also, “look at what they’re actually doing in terms of legislation and implementation.” And I also do one… I don’t use the textbook anymore, but it’s something I used years ago that I retained the content in a module, which is to look at the concerns of any country in terms of every country’s concern with prosperity, security, and stability. And so I asked them to just put out an infographic that asks them to assess threats to those things, and looking at their country. And so it’s a little bit on the current events side, but it’s focused for them in terms of every country needs to be aware of its security… what’s happening?

Rebecca: You’ve hinted that students are really engaged in these assignments because of discussion forums. But can you elaborate on just the general sense of what students have been able to accomplish by shifting to this format, and also their engagement with one another?

Michelle: I think, several things, some of which I’ve mentioned. So the formative assessment part, I think they gain a lot from not just getting comments at the end of the semester, oftentimes handed in that last week. And then, frankly, I don’t expect that every student has even read my comments at that point. So just having the opportunity to fully engage throughout the entire semester, and so we’re not also dealing with, “Oh, I have three days before this paper is due. So let me think about it today, make a plan to think about it more tomorrow, [LAUGHTER] and then maybe I’ll do it the next day before it’s actually due.” And not that that’s all students, but I know when I was an undergrad that happened to me. So I think that just slowing them down, allowing them space to actually think about their country in multiple ways, and not have it be “Okay, we’re all working to doing your own country study at the end of the semester,” but having them actually engaged the whole time, I think is of great benefit to them. And I think they learn a lot more. I think that taking it in these smaller chunks, actually has made them research a little bit deeper on each of the topics because they’re not overwhelmed with “Okay, I’ve got all of these topics now to cover in this country study.” And so they tend to just do it in a way that is most efficient for them, let’s say. And so I think this gives them that way to back up a little bit and actually enjoy the process of learning rather than the process of an assignment only.

John: I think you’ve made a really good point there about students’ tendency to procrastinate. I was thinking back to when I was an undergraduate, and actually a graduate student as well. And I can’t recall a single time when I didn’t start writing the paper the day before it was due. I spent a lot of time gathering materials, taking notes, and organizing and putting an outline together, but the writing generally took place in just one day. So, [LAUGHTER] this is forcing students to engage in a little bit more spaced practice, to engage with the material regularly, and I think there’s a lot of benefits over that rather than the traditional way of just rushing through and getting it done right before it’s due.

Michelle: Yeah, exactly. And I’ve adopted that in other classes too, the idea of doing a lot of prewriting throughout the semester. So I’m a big believer in the prewriting process, and it’s very valuable, and it is doing work. But when I mentioned earlier, sort of they spend the day thinking about it, I’m thinking more of myself as an undergraduate thinking about not the assignment, but thinking about the fact I have to do an assignment.[LAUGHTER] So it’s more anxiety, not so much productive work on the content of what I’m doing. And so yeah, the prewriting work for any assignment is really great. So I’ve been trying to shift that mindset. I also teach a research methods course in my department and I teach our senior seminar. And so the senior seminar, especially, just making it about an entire semester-long process, I think, has taken a lot of pressure off of them. They seem a little relieved by the end, it’s not as daunting,

Rebecca: It definitely seems like it deepens the study and the engagement, not just because it’s broken into smaller pieces, but because it’s a consistent country the whole time and because it’s a consistent kind of assignment the whole time, it really provides a structure to be able to do that. I’m curious whether or not any of your other colleagues have also shifted to doing assignments in this format, or other similar kinds of strategies.

Michelle: I can’t really say. I know that in my department, we’ve talked about them a lot. And so I think people are adapting parts of them. I don’t think anybody does it the full way that I’ve described in terms of the full semester, but we have talked about just different ways of communicating. I think that’s going to shift again, obviously, AI is something that everybody is grappling with how to best use. So I think there’ll be some shifts coming. But I can tell you that I was Interim Director of the Center for Excellence in Teaching and Learning, and now I’m a faculty fellow there. And I’ve talked to a lot of people across campus, and some of our off-campus partnerships about this assignment, because I get really excited about it. If you haven’t noticed, this is my favorite thing. And so I’ve talked to a lot of people, and I just can’t be sure how many have actually applied some of these things. But they certainly seem interested, and they’re certainly searching out the same types of things I was searching out: how to make things, first of all, more authentic for the students, how to engage the students more with themselves in an asynchronous class or with other students. Even in a synchronous class or on-the-ground class, you could still have the same assignments. You might want to introduce them differently, you might want the discussions to take place in the classroom, rather than obviously online, because you could engage in person that way. So you can do kind of a round robin around the room, maybe… any of those techniques. But I also let the students know that, for them, they will become a mini expert by the end on that particular country. So they should all understand the topic areas but. becoming an expert on the individual countries that’s all their own. And so I hope that colleagues, no matter what field they’re in, can find some way to utilize that. Otherwise, they’re probably just a little sick of hearing me talk about it. [LAUGHTER]

John: Speaking of fellow faculty, has anyone else shifted to using infographics at your institution as a result of your work with us?

Michelle: Yeah, I can’t be quite sure. But I do think that in talking to a lot of faculty that they are shifting modes of communication for students, and I have talked to people who are much more expert at creating infographics themselves. You can do it very, basically, but you can also get really creative and high level with it. And so I oftentimes collect infographics I see for other fields, so I share those with faculty. I’m not sure. It’s one of those things that I so appreciate a podcast like this because sometimes teaching can be a lonely endeavor. We’re a little bit maybe shy about sharing what we do, because we’re afraid it’s not measuring up necessarily to what others are doing. So it’s a good question, though, and I think that that’s something that maybe I want to engage… maybe in a workshop in the future on campus and bringing faculty in and seeing who’s doing this type of thing, or maybe another brand new idea that’s even better.

Rebecca: I think one question that often comes up from faculty when we’re talking about kind of alternative formats or unessays or other [LAUGHTER] ways of not necessarily doing a more traditional essay kind of assignment, is how you evaluate the work. So I wonder if you can talk a little bit about how you’re evaluating these assignments, just to give faculty a sense of what that might look like and what that feels like in comparison to a more traditional kind of assignment?

Michelle: Yes, that’s a great question. I’ve talked in many ways about why I did these assignments in terms of just making it a more engaging process. But in terms of measurable, assessable goals, I do use rubrics, and have laid out exactly what skills I’m assessing. So I do have a section of the rubric that is based on the skills of communication. I have a very low stakes, low number line for things just like visual engagement, because I don’t want it to be students worried about the visual engagement when they need to learn the political science concepts. but I want to let them know that that’s important. So I have in the rubric descriptions in each of the columns about visually engaging: Does it bring the reader in? Is it easy to read? So I have that skill section, I have a section specifically on the content itself: Do they understand the concepts? Are they using the language that we use in political science? And then I also have sections for pretty typical traditional looking at their writing and seeing if it’s appropriate writing levels, tell them take a look at other people’s infographics… go out in the world and look at the bottom, there will be the sources, you need to include those, you need to not leave those off. So even practice that they would get in papers about citing things, if they put a graph in, you still have to say where this graph came from. So the practice that they would normally get are the same skills. And so I just lay it out in a rubric for them, as I do with most of my assignments. Of course, the rubric is not secret to the students, that goes out with the very first assignment: this is what you should be looking for. I’m going to use it for grading, you can simply use it as a checklist. You can use it to evaluate other students and give them feedback and tell them where you think there could be improvement. And so I think that the assessment itself is pretty typical to what I would normally do when thinking about an assignment: What are my goals for the course? How does this assignment help to meet those goals? And then how do I convey to the students whether or not they’re meeting those goals? Or essentially where they’re at in the process of becoming expert in that goal.

John: Do you use one rubric for all the assignments? Or are there separate ones for each of the infographic assignments?

Michelle: So I’m a big believer in constructing the outline once and then using it, but having different content in it. And so I use the same basic rubric for each of the assignments. And then I give the feedback in that content area really is the main portion. In the content area, I will make some changes to what they need to do specific for that assignment. So if it’s political parties, I would have expectations about what types of things they’d understand about political parties. And so that will change but my basic outline of what that looks like. And I’m a big believer in that kind of consistency in everything. So I can tell you I do that also with just how I set up the modules. And so my modules always have the same outline. There’s an intro with the learning outcomes for that specific module: what can you expect to learn this week, then the section with textbook readings. So this is what you’re going to dig into and read in a very traditional way. Then there’s a whole separate content section, which might be a video I’ve created, it might be videos that I found online that they should be engaging in. And then I usually have a pre-test and post-test quiz, and then the assignment for the week. And in this case, always associated with that assignment discussion. And then always at the end, I have a section, another discussion, which is I just call “Ask the professor,” And it’s another space for it. I always have to tell them every time I put the same heading and I say, “Don’t ask personal questions here, [LAUGHTER] but if you have a question that everyone can benefit from on the topic this week.” So that’s just an example of how I feel that it just makes it easier for me to focus on what’s in each of those boxes, rather than recreating the format. And I think students really take a lot of comfort in knowing after a couple of week: this is the rhythm, this is what I’m expected to do. And so now they’re focused on different content, but they don’t have to relearn a different set of skills. So going back to your question about the rubric, and is it the same? I do that because then they get used to what skills they’re bringing, and then they can hone that same skill over time, they can go back and look at their rubric grading and see their improvement over time. And I’m also a liberal user of the comment section on rubrics. I know that a lot of people tout rubrics for the time-saving part of it. So oh, you can check these boxes, and it makes it easier. And there’s some of that. But I can’t stop myself from also than opening up that box and just writing a ton of things to them. So I think that, again, it’s one of those things where the rubric itself becomes a really good outline. And then it’s just a matter of what I choose to put into my grading when I do the rubric.

Rebecca: High structure is always helpful for folks, that’s for sure.

Michelle: That’s very reassuring.

Rebecca: I feel like there’s a story behind why infographics as opposed to something else. What led you to infographics as opposed to some other alternative format?

Michelle: So like I said, I’ve been at this 27 years, so maybe I’m just getting old, but I feel myself saying in my head: “Well, kids nowadays don’t have the attention span for writing a 20-page paper.” And that’s not necessarily true. But as I talked about before, maybe they just have to be taken through that process. It can’t be just one big thing. And that was long before the internet, people talking about decreased attention spans. But I do think there’s something to matching up with the way in which they use visuals in their daily life. So they utilize memes, they look at infographics in various formats, I find a lot of them in social media in different areas. So I try to structure discussions in a way that is very much like the way they’re already engaging on social media. So yeah, the story is really how could I find a way that it’s something that their mind is already used to, that they can then apply it in this way. And also the other story, my husband is the County Administrator here in our county. And so I was in his office one day, I was the Assistant County Administrator and I was in his office one day and from the League of Minnesota cities, they had an infographic, a stack of them on the front table about: what does county government do? And that was really the last push I needed because it was so well done. And I thought, “Okay, this is what we need, people need to understand what levels of government do.” People often come to the county with situations that have nothing to do with what counties are actually able to do for them, and so what does your county do? What can it do for you? And I just thought, “How brilliant is that? To put it in that way.” And so I just really was sold on on the idea that infographics can really be fairly deep teaching, if they’re done well. We get so used to the academy and everything has to be these long papers, but rather than bemoaning the fact that the attention span has gone away, I’m just finding a way to adapt to that. So how do students engage with each other? And let’s make that happen.

John: And I would imagine students are more likely in the future to be creating infographics than they are to be writing 20-page papers unless they’re going on to graduate study.

Michelle: Yes, absolutely. That was the other part of it is that this wasn’t really authentic… the 20 page paper… particularly in that intro, liberal education, general education course. That was not authentic to what they were going to be doing. And I felt that in political science, if they were going to be political science majors, they would get that deep research component in other courses and upper-level courses in their capstone. And so I felt that it wasn’t necessary to put them through a 20-page paper. And the reason I keep saying 20 pages is because I realized that the amount of content you need in the country study is pretty lengthy to really understand the country, and so it really was about 20 pages, I just felt that that wasn’t engaging to people, no one was engaging each other with a 20-page paper. And so it had to be something that you could look at more quickly, you have to have the skill of condensing the material, you really had to be really pithy with how you were presenting the material so that someone can get it in a very quick way. And so I feel that allowed them to engage. And then even if you assigned a student to engage with another student’s 20-page paper, pair them off, have them do peer evaluation, they weren’t going to do more than one at that length. Maybe if you cut it down to a 10-page paper, you could have them have a group where they exchange it among three or four people, but that’s still not as big as I’m able to get by having everybody in a class post their infographics so that you can all see huge variety of work in the span of scrolling through normal social media. And of course, I’m asking to do a little bit more, but in some ways, it’s almost that they don’t even notice that part of it because they’re just scrolling through and saying: “oh…” Comments that I get most on the first assignment have to do with “Oh, I’m so glad you chose to do that country, I’ve always wanted to go there. Have you ever been? …because it looks so interesting.” So it doesn’t start out as this deep political discussion. But because of those comments in the first week, I’m convinced of the engagement that starts to come from it. One part that I didn’t mention is I also put in the directions, that if they want to add something, like a fun fact, about the country on any of these infographics, they should, and I’m surprised at how many students then choose to do that on every single one of the infographics at the bottom. There’s a fun fact. And I’ve been amazed by what they’ve come up with. There are things that I don’t know about these countries. I thought, “this is fun.” So I do think that students take to it, they want to show other students their country, they want to say, “Hey, I learned this thing. Isn’t this cool?” And then I get to learn things I didn’t know.

Rebecca: Sounds like a great way to build community and to have a first interaction with a discipline, which sometimes can be off putting. And this sounds like a really positive way to get people interested in a discipline as well.

Michelle: Yeah, I think so, and I hope so. I hope that that does make students more likely to maybe take an upper level, look at us as a minor or major. And I will say, talking so positively about this assignment, because it’s worked so well. But I can also talk to you in a podcast about all the assignments that haven’t worked well, that I don’t get this excited about, that I can’t say were this level of success. And so when we were talking about coming on the podcast, I wanted to highlight this one, because of course, it’s the success. I wanted to show what has actually worked. But, of course, there’s a lot of ones that haven’t. So I do feel that I’ve hit on something here that can be very useful, that I’ve been really satisfied with, and my students seem to be really satisfied with.

John: We always end with a question. What’s next?

Michelle: So that is a great question. I’ve been thinking about that for a while. How do I take this to the next level? So I think one thing that’s next is sharing this out more. We’ve talked a few times about looking at other faculty and seeing if they would adopt something similar to this. So I think that’s one aspect. I think another aspect is to just continue to refine the assignment in a way that gets the most student engagement. I’d also like to see these assignments, if we can have them maybe printed for my asynchronous online classes. This particular class I teach mainly online now, I rarely teach it in the classroom another colleague has those sections. And so, right now, it’s simply archived either as the students own individual file, or in D2 L. And so I’d like to think about ways in which maybe we can archive them, maybe print out them in poster form and have them in the department so that people can maybe again, engage, get interested, a student walking by who’s not in political science who’s hanging out in the hallway waiting for their next class to begin, and starts reading the stuff on the walls and start saying, “Oh, that’s kind of interesting,” and maybe starts to get engaged. So looking at it outside of the particular classroom, I think would be a really exciting way to go with it.

John: Well, thank you. I’ve really enjoyed hearing more about these assignments. And I think it’s something that could work in many disciplines.

Michelle: Yeah, I hope so. And exciting for me would be now to sit back and kind of watch what other people do with it. And oh, okay. I hadn’t thought of that. That’s really cool. That’s always the most exciting part for me from the CETL angle is really just seeing how people take off with something in a way that I wouldn’t expect. And it’s been a pleasure talking with both of you.

Rebecca: I always love a good conversation about interesting assignments. So thanks for chatting with us and sharing in depth how the assignments work over the course of the semester.

Michelle: You’re welcome. My pleasure.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

338. Diversifying the Education Pipeline

Diversifying various fields and disciplines requires intentional work to create and support a pipeline of practitioners. In this episode, Laura Spenceley joins us to discuss specific initiatives to increase inclusion in the PK-12 sector. Laura is the Dean of the School of Education here at SUNY Oswego. She is an Impact Academy Fellow through the national non-profit organization Deans for Impact which works to strengthen and diversify the educator workforce.

Transcript

John: Diversifying various fields and disciplines requires intentional work to create and support a pipeline of practitioners. In this episode, we talk about specific initiatives to increase inclusion in the PK-12 sector

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Laura Spenceley. Laura is the Dean of the School of Education here at SUNY Oswego. She is an Impact Academy Fellow through the national non-profit organization Deans for Impact which works to strengthen and diversify the educator workforce. Welcome, Laura.

Laura: So great to be with you two today. I’m really excited for our conversation.

John: It’s overdue. We’re really happy to be talking to you today. And we’re talking to you from a new room which the School of Education has graciously loaned to us because we lost our old recording space, and we’re now able to keep the podcast going. So thank you.

Laura: You’re most welcome. And truly, I can’t think of a more symbiotic relationship between Tea for Teaching, the opportunity to host you here in the School of Education, and continue this great work and digging into pedagogy issues across the education continuum. And having you right here in-house makes a lot of sense.

John: Our teas today are:… Laura, are you drinking any tea?

Laura: I am not a tea drinker. But I do have a robust French roast in my mug today.

Rebecca: A popular flavor of tea, as we’ve noted multiple times here on the podcast. I have Hunan Jig today, John, I thought it was a good celebratory sounding tea.

John: And I have a spring cherry green tea in a brand new School of Education mug, which I just received today. Thank you.

Laura: You’re welcome.

Rebecca: In your role as the Dean of the School of Education, you’re involved in the entire P-20 pathway. The School of Education has received two large grants to help support students pursuing careers in education. Could you tell us a little bit about these grants?

Laura: I’m thrilled to talk more about these opportunities. And these two projects are our teacher opportunity core grant, that’s our TOC grant and our Cultivating Representation in School Psychology grant. Both of these grants are, as their name may imply, designed to help us recruit, support, retain and move through into careers, teachers and school psychologists from a wide range of backgrounds. It’s a well known fact across most educational spaces that folks who are delivering instruction or providing support services don’t always represent the constituent student populations with whom they work. And each of these programs are designed to 1. help ensure that we have a representative and diverse education workforce. But more importantly than that, they’re also designed to ensure that as we recruit and matriculate students into our teacher preparation programs and school psychology program, that we’re investing in those candidates along the way, recognizing that students from historically marginalized groups are often more likely to have challenges in persisting through their academic programs, both undergraduate and graduate. So each of these programs has some similar foundational elements. That includes financial support along the continuum of their educational program, again, undergraduate or graduate, support for things like transportation, professional development opportunities. And those professional development opportunities are really designed to empower our students from those historically marginalized or underrepresented spaces in their fields with contemporary activities that will help them address systemic discrepancies in schools. So making classrooms more responsive to the needs of the multicultural communities in which they reside, helping ensure that our future psychologists and educators have opportunities to work with people along the way that look like them, that share their lived experiences, and build that really continuum of care around their success. It’s also a great opportunity for us as educators, leaders, teachers, to be invested in those spaces with them and hearing from their mouths, what their challenges are, what their opportunities are in districts and really working to empower them for a career in education.

John: And the School of Education has always been engaged in working with students from all these marginalized groups that you’ve mentioned. What other types of support, though, have you been working on developing in the School of Ed for students who are first-gen students, Pell-eligible students, or from historically minoritized groups?

Laura: It’s a great question, John, and I’m so thankful to have the opportunity to chat about it because I think sometimes, as is human nature, we want to create responses or programming from a perspective of “all first gen-students might need this thing” or “all Pell-eligible students might need this thing” and we’ve been certainly sensitive to the realities that those populations experience, but also have tried to take a step back and take a more nuanced approach to that work. I’ve had the opportunity to be in the Dean’s position for about three years now. And when I came in the door, it was the year after the global pandemic. And so, so much of what we knew here on a college campus and in our P-12 spaces in the community had just been turned on its head. So when I came into the dean’s office, one of the things that was really important was to hear from our students, our candidates pursuing their degrees, what are the challenges that you’re experiencing? And I need to give a shout out to Dr. Nicole Brown, our director of our clinical practice and partnerships office, she really had this vision of surveying students, engaging in some listening sessions with those students. And we took the opportunity to deploy a survey about how students were experiencing their programs, but also the challenges that they were experiencing. And those data were humbling, to say the least. We learned back in 2021-22, about one in five of our students was having to make the choice between eating and paying for gas to get to their placements. About one in four of our students had to go without a meal at a certain point so that they would be able to engage. We recognize, and this was not a new problem, but just how challenging transportation to and from their clinical sites was becoming particularly in a time where we were very sensitive to being in enclosed spaces. And so we took that opportunity to reflect on the ways we weren’t just to use quotes, “utilizing the resources,” like our teacher opportunity core program, but to really recalibrate the way we were engaging with all students. And so what that looks like right now is we’ve really taken an approach to engage with our students from the time they’re prospective students all the way through graduation and career placement. And so what that has looked like is we’ve had active faculty and administrator presence at recruitment events, helping prospective students and their families understand not just the phenomenal financial aid resources available to them, but demystifying grant programs like the Teach Grant program, like public service loan forgiveness, which is available to many of our graduates after a period of time, it’s also about helping those families understand what contemporary classrooms and contemporary educational careers look like. There’s so much animus in public media and in the dialogue around what’s going on in schools and giving our prospective students the opportunity to hear from our current students and our partners, what they can actually expect, not just out of their degree program, but the field, has been really impactful. On a more nuanced level, as a result of that survey, we have utilized a range of resources to help offset some of those specific concerns regarding transportation, costs related to certification, and food instability. I do need to call out our amazing student affairs team on campus that has made enormous investments in this space as well. And I think always connecting students to the most local resources has been a really effective way to ensure they can take advantage of those supports in a way that makes sense to them. For us, I was lucky enough to inherit a 12-passenger van that had been purchased right before the pandemic, and we’ve taken the opportunity to hire student drivers. We offer daily routes to and from the Syracuse area at no cost to our students, leveraging the generosity of our alumni and donors who have invested in the School of Education, we have made a commitment for any candidate that needs to get to or from their site and can’t use the van, that we’ve created an Uber voucher system. And so we’ve been able to put those funds directly in front of students. We’ve expanded our supports for certification exams and fingerprinting through really simply asking students: “Are these barriers?” and creating a process again, through our foundation funds, that we’ve been able to offset those costs in time of need. And then lastly, I think another part of this is helping our faculty understand the realities that our candidates are experiencing as they move through their degree programs. Bringing those data back from not just the survey but listening sessions we continue to offer as an open forum for students to share their concerns, bringing that back to our faculty and helping them understand in both a formative and accurate sort of way, what the experiences of our students are in looking to make changes, whether it’s to our core sequences, the way that we administer fees, encouraging open-access resources, building resource libraries within our departments, so that we can continue to wrap our arms around students, give them a great experience, and also ensure that something like a meal isn’t getting in the way of their ability to get to and from their site.

John: And one thing we should note is that Syracuse is about 40 miles or so from here. So the commute time for a student would probably be about 45 minutes to 55 minutes, depending on where the school is located. So transportation would be relatively expensive for students.

Laura: Yes, and it’s such a good point to make, and I know we’re likely to talk about partnerships later, being where we are here in Oswego, we are lucky enough to have amazing partners along the I-81 corridor from Syracuse to Oswego, almost to Utica, almost to Rochester. And so it is not unusual that our candidates are driving up to an hour each way. So absolutely.

Rebecca: And it’s a problem that campuses that are in a more rural context experience when they’re doing any kind of placements, whether it’s internships, student teaching, or service learning, or anything like that. It’s very different than an urban context.

Laura: Absolutely. And I think we’re really committed, we embrace the idea that we are a rural educator preparation program, while also embracing the fact we have large urban partners just down the road. And we see that as an opportunity to diversify our candidates’ experience and give them a broad sampling of school structures, organizations, what’s possible, both in large districts, small and the in-betweens. And so that’s been a really critical part of our preparation framework for many, many years here in the School of Education.

Rebecca: It’s always great to hear stories, and we’ve heard more of these since COVID, of schools really paying attention to the actual needs of students and what barriers they’re facing, and really being responsive to them and finding ways to solve the barriers that they’re facing. Because often, the barriers are great, but often there are solutions that we can put into place that aren’t actually that complicated when we get down to what they are.

Laura: Rebecca, I love so much how you framed that, because one of the driving mantras that I often remind myself is we can solve problems if we choose to. And I think to solve problems, we need to be able to describe the problem, we need to understand it from a compassionate perspective. Recognizing every candidate that declares a School of Ed major sees themselves as a future educator in one form or another, and partnering with them, to actualize that vision is the best part of my job.

Rebecca: Well, and we would hope they want to do the same thing for their students. So modeling what we’re hoping to bring into the land of education is certainly a good place to be. We talked before COVID, with the School of Education about some of the really great partnerships you all have both locally and regionally, both as school districts and community organizations. Can you talk a little bit more about some of the outreach you’re continuing to do in those spaces?

Laura: Absolutely. And frankly, we would not be a school of education without partners. Every program housed within the School of Ed requires an applied learning experience. And so even our non-teacher preparation programs in counseling psychology and management programs require some sort of applied experience. And so strong relationships and a strong understanding of the context for our partners is really important to our success and our candidates’ success. So I’ll talk about a few examples since there are many day to day here in the School of Education. For context, in 2020-23, for example, we placed more than 1000 candidates across I believe 246 different settings, kindergarten all the way up through high school. Don’t quote me on the number, but I think I’m close. But we have made a couple of purposeful efforts over the last few years. And in particular around our professional development schools. These are schools where we have an agreement to focus professional development activities for our pre-service teachers, in-service teachers, so folks who are classroom teachers in that district, and align that with our faculty areas of expertise to create what are called that third space for learning, where we were bringing in a range of expertise and perspectives. Here in Oswego County, we have professional development school partnerships with six of our nine local districts. And those partnerships have focused on strengthening support for technology and STEM-oriented initiatives at the junior high and high school level. In another district, it’s included a dedicated focus on early elementary school literacy outcomes and improving the way we’re preparing kids to read and write. Other districts have focused on more systems-level change with strengthening the way that we identify students who are struggling before that becomes a significant impediment to their learning, and organizing teacher and leader responses to ensure that those processes are unbiased, that we’re making decisions that help us put resources around what kids need to thrive. But that’s just a sampling of the types of partnerships that we have with local districts. I think, given the fact that education continues to be a domain where the public dialogue is not always framed as positive, partnerships with our local practitioners and districts, give us opportunities to 1. change that narrative and help people in our communities understand the value that schools can bring to that community. And Rebecca, you mentioned the way that, coming through the pandemic, schools are really regarded as kind of community centers. And I think we’ve seen those types of initiatives be really effective, whether that is including opportunities for families to access medical care, dental care, social services and supports when they’re in district, we have something to offer to that conversation because of the range of our preparation programs here on campus. And so we really tried to engage with our partners in the P-12 sector to ensure that we are operating as a true continuum of care for our communities.

John: During the pandemic, there were some fairly substantial learning losses that disproportionately affected students from low-income households, and low-income school districts. And a study was just released by WICHE in February of 2024, which found that those learning losses are still persisting, which means the students who are teaching in schools are going to face students with a fairly wide variance in their prior preparation. What types of things do you do to help prepare students to address the diversity of student backgrounds in terms of how much they come into the classes with?

Laura: Coming through the pandemic, one of the things we were reminded of was the power of a direct positive relationship between a student and an adult in the building, that can look like a wide range of folks, coaches, teachers, mentors, administrators, volunteers. But that’s something we have continued to embed in our curriculum and helping our candidates understand that day to day, the way that we make students feel is often the thing that will be really critical and helping ensure that they walk through the door the next morning, the next day, or ask for help when they’re having a tough situation. And so empowering our students to respect the dignity of people’s lived experiences, we embed before they move into their clinical experiences in schools, quite a bit of reflection, self-reflection on their own lived experiences, what education has meant for them, the spaces where they have been privileged or perhaps oppressed through educational structures, protocols or policies, and building that capacity of our students to both identify and address bias or embedded bias in the spaces where they work. Getting down into the curricular focus, it is frankly staggering to see how children have been impacted, particularly in the early elementary levels and the older adolescence ages when we think about both academic skills, but socioemotional skills, wellbeing, and I think one of the things that I try to celebrate is the resilience that’s been built into this generation of learners. On the other hand, that comes with a certain amount of exhaustion, anxiety, uncertainty about their future, and so coming back to that element of relationship building, it’s so key. At the curricular level, there is an enormous focus here in New York on the ways that we can do a better job in preparing children to be literate, contributing members of a just society. So right now in New York, there’s a lot of focus on the science of reading. And I want to be really clear to listeners, the science of reading isn’t about a curriculum, it is about embracing the empirical evidence towards a systemic approach to the instruction of reading, writing, and literacy across grades. There’s an enormous amount of energy around this work right now recognizing that we can do better in preparing kids to be literate. And I’m starting to see an equal attention to a focus on math and frankly, STEM preparation. Here in our community in central New York industry has returned with a bang. And there are wonderful opportunities for employment for students with a wide range of educational experiences starting right out of high school for gainful and lifelong employment in these sectors. But what we’ve heard is that math proficiency is often a barrier for those students. And so trying to incorporate elements of curiosity and creativity, bringing science, technology, and math into creative spaces, like art, design, dance, helping students understand that math and science, quite frankly, isn’t just about memorizing equations, or knowing your multiplication tables. But there are a range of application of math, science in our day-to-day lives. When I think about when I was a kid, we had a great lesson on fractals, and we went outside and we found fractals in nature. I remember it to this day, almost like it was yesterday, trying to engage students in their learning, that incorporates their lived experiences, having access to textbooks, readings that reflect their family structures, their linguistic background, the realities of their lives. And so our students ,when they move into those first teaching positions, carrying with them that, I’ll go back to respect for the dignity of all persons, that recognition that kids carry much more with them into a classroom than a backpack, and finding ways that we can empower their learning not just through curriculum, but through curriculum that is delivered at their level, that celebrates individual students growth, that isn’t anchored in some specific outcome, but recognizes individual student’ s growth as meaningful. And I’m really proud to say our data suggests our students do move into their first teaching careers with those skills. And I think the next phase of the conversation starts to focus on how can we, as an educator preparation program, be part of the community in delivering professional development? How can we bring the great work that’s going on in schools to our faculty and have that kind of recursive flow of information, and also ensuring that we are preparing school leaders to create conditions that help us achieve all of these goals. And with our programming here in the School of Education, we’re uniquely positioned because we offer those range of preparation programs and have the luxury of great relationships with partners.

Rebecca: We’ve talked a little bit about learning losses in K-12. But obviously, that has an impact on our college students as well. They also survived the pandemic and are here. What are some of the things that your faculty are doing to help these candidates feel like they have what they need to go into the community having maybe had some experiences of remote learning or challenges with some of these subjects too?

Laura: This is a really important question. And I think it’s one I’ll both answer from the school of education perspective, but also the broader campus perspective, frankly, because we don’t do this work in a vacuum or in a silo. And so I’ll pick up on some of the elements of the last answer, which is making sure that we on campus are framing the opportunity to serve our students as a positive one. We have an opportunity to meet students where they are and that will look different for different students. On the other hand, I think our faculty have been eager to try to crack this code, so to speak, on bridging our lived experiences in K through 12 and transition to college through the framework of the contemporary student experience and those are not parallel for many of us or really even comparable. I have been trying to, as dean of the school, make sure that we are approaching this from a perspective of strength. What students do come to us with us is a resilience that students of prior generations may not have. They come to us with skills and knowledge of technology and how to leverage it in a way that students didn’t in the past. And frankly, it was because they had to learn how to use that technology. Students coming to our campus today have access to more information at their fingertips than any generation of student has ever had in the past. And where we have opportunities is to help students both identify where they may have gaps, whether in their knowledge or experience, put those experiences in front of our candidates, and also help them navigate this world where it’s not always easy to identify what is good information and bad information. Now, let’s drill into some more specific examples there. I think one of the most critical things that we have done in the last few years and again, I’ll give kudos where they’re due to our student orientation groups. We have been embedding ourselves into some of those opening week sessions, where we’ve had the opportunity to talk to our future majors and undeclared students around college- level expectations. And while that might sound silly, I think when students can hear from us, from me, from my faculty colleagues, our stories that many of us are first-gen, were Pell-eligible, have student loans, worked while we were in college, struggled to purchase textbooks, come from different backgrounds, are new to the state or the country. I think that goes a long way as a formative first step in building that trust in the relationship with our candidates. It also gives us the opportunity to begin to plant those seeds around what supports are available to them. Whether that is to help them address their mental and physical health needs, their academic needs through our tutoring center, our office of learning services, the embedded tutoring services, and also come to see us as people that are here to help support them and what their vision for their success looks like. I had a young person come up at the end of our first orientation and said, “I’ve never had someone like you say, you walked in the same path as I have.” And it was really a moment where I understood that coming to the table and being authentic with our students is so critical at this point. The other piece of it, John and Rebecca, is recalibrating the way that we orient our classrooms to continue to give students frameworks for utilizing technology and AI and the wide range of information, also helping them become critical consumers of that information, providing formative feedback to their assignments in real time, flipping our classrooms and allowing students to lead the conversation. I think providing choice as much as possible around what literature students choose to present in classes, for example, have been ways that we have said to students, both overtly and more discreetly, you matter, your story matters and you and my classroom matters. I’ll also share we have been taking a look at the way we offer our curriculum to try to identify, and again as part of a larger campus focus on retention and success for our first-year students, where are the pain points, so to speak, in their degree program, particularly for transfer students, we’ve heard that there isn’t always as seamless of a transition to campus as we may assume. And so really starting to tackle some changes in our course sequence to ensure that our academic programs are set up in a structured sequential way that also gives enough flexibility for students to pursue some of their areas of interest, be student athletes ,be club leaders, or frankly, support their family, which many of our students need to do.

John: Recently, we’ve seen a lot of attacks on education at all levels, with books being banned in the classroom, removed from libraries, and restrictions on the types of topics and the types of history that can be taught in classes. What are some of the things that you do in the School of Education to help prepare students for a world in which they may be facing more challenges from parents and political pressures?

Laura: This is a critically important question and will be critically important action. I hope I could say in the next year, but I have a feeling this will be a long road. Too often I hear colleagues, neighbors, friends say, “Well, Laura, at least we’re in New York, you don’t have to deal with this in the same way.” And while it is true, I am proud to be in a state that has been very overt about people’s lived experience, respecting the civil rights of all persons here in the state. It is also the case that that can change very quickly in an election cycle. And frankly, we have seen that across the country. I’m particularly concerned right now about the way that civil rights discussions have been framed. And there are some cases in litigation right now around bullying initiatives and carving out exclusions or exceptions in those bullying protocols or policies that would allow, for example, trans students to be targeted without recourse. We see that replicated in both sports spaces, but also classroom spaces. I’m deeply troubled that this is a place that we’re going in a country that was frankly built upon public education and access. And we haven’t always gotten that access piece right. Many of our school policies and procedures have been built on an exclusionary clause rather than an inclusionary mission. On the other hand, we’ve made such great gains over the last 10 to 20 years in ensuring that schools are more responsive, supportive, dignifying places. I’m really troubled by this work. For our students, we have a commitment to social justice embedded in the framework of the School of Education. And that is more than transactional or it goes beyond the words social justice. Our curriculum prior to students going into schools is focused on those ways that schools have been exclusive spaces, have not always dignified the realities of the people who walk through the doors. And so for our students, what we tried to do is help them understand the threats or attacks on people’s civil rights or their histories is not new. On the other hand, it’s taken a very different tone in the way that it’s being implemented. So 1., it’s giving them the tools in knowledge and understanding of history to understand schools don’t have to be exclusive places, schools can be responsive, welcoming, encouraging spaces for all students from all backgrounds. It’s also about helping students understand that I want to be a teacher, I don’t want to be a politician is not a reality in which they will live as an educator. We try to empower them without encouraging them to engage in any particular behavior, to stay attuned to the realities of how legislative efforts, school board votes, and other sorts of community oriented book bans and dialogues might impact their ability to teach a particular lesson, utilize a particular academic resource. So you may have seen coming out of Florida, among other things in the last few weeks, a school district teacher had thought that it was prudent to send home a permission slip to alert the parents, that their children were going to be exposed to a black author. It wasn’t about the content of the book, it was simply the fact that they were going to be reading a book from an African- American author. I can’t think of a better example to highlight the importance that an apolitical stance is not one that classroom teachers can take, that school leaders can take. And that’s not about voting for a particular person or a particular party. But it is standing up to ensure as educational experts who have preparation, expertise, and insights that the community may not, to share them and be vocal about the ways that diverse learning materials, accurate historical representations of the world are critical towards ensuring that we do not replicate them. And so I am both incredibly disheartened at the turn that this discourse has taken. I’m also incredibly optimistic when I look at how young people, how educators, how experts have shown up and linked arms to say: “This is not right. We are not going to correct the wrongs of our history by limiting access to that history or by building smaller tables with fewer voices.” And so I tried to stay in that headspace as much as possible.

Rebecca: Definitely some good reminders and things for all of us to be thinking about in our role as educators. So we always ramp up by asking what’s next?

Laura: I love that question, because I could probably read you a laundry list of the things that we’re thinking about in our School of Education. Right now, here in New York, there’s a lot of robust conversation around, what does a high-quality teacher look like? What are they ready to do in their classrooms? And there’s a lot of engagement there. And again, this relates to helping the public understand not just what we do day to day or what the outcomes are, but the opportunity space towards preparing children for lifetimes of success. Here in the School of Education, we’ve got a range of new programs that we’re lifting in the fall of 2024, including our new master’s degree in behavioral health and wellness. And frankly, I think we have a range of new programs. As we talked about industry thriving here in central New York. There’s also a lot of energy in the school of education around our career and technical educator preparation programs, and of course, our technology education programs, some great investments in our lab spaces, looking at opportunities to do more in partnership in and with local districts to ensure, as we were talking about kids across Oswego County, have great opportunities to engage with science and technology and math in ways that bring them out of the classroom and into the world and creative spaces. So I’m really excited to see what the year ahead is going to look like for us.

Rebecca: Well, thank you so much for joining us, Dean Spenceley and for our wonderful space.

John: …and we’ll be talking to you much more in the future.

Laura: I look forward to the opportunity. Thank you both.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

326. UDL in Action

Universal design for learning, or UDL, is a framework to help us design more equitable learning experiences. In this episode, Lillian Nave joins us to discuss how she has implemented a UDL approach in her first-year seminar course. Lillian is the Coordinator of the Center for Excellence in Teaching and Learning for Student Success at the Appalachian State University Hickory Campus and a senior lecturer in a first-year seminar course at Appalachian State University. She is also the host of the ThinkUDL podcast. She is the recipient of several teaching awards and often serves as an invited speaker on UDL issues.

Show Notes

Transcript

John: Universal design for learning, or UDL, is a framework to help us design more equitable learning experiences. In this episode, we discuss how one faculty member has implemented a UDL approach in a first-year course.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Lilian Nave. Lillian is the Coordinator of the Center for Excellence in Teaching and Learning for Student Success at the Appalachian State University Hickory Campus and a senior lecturer in a first-year seminar course at Appalachian State University. She is also the host of the ThinkUDL podcast. She is the recipient of several teaching awards and often serves as an invited speaker on UDL issues. Welcome, Lillian.

Lillian: Thank you very much. And I applaud you on all of the monikers that I currently have attached to my name that often don’t make sense. So, very good. And of course you said Appalachian in the way we say it down here in North Carolina and I don’t have to throw an apple at you. [LAUGHTER]

John: Well, I’ve heard it so many times on your podcast that I wanted to be sure we got that down.

Rebecca: Today’s teas are:… Lillian, are you drinking tea today?

Lillian: I am and I brewed it specially for our interview today, it is Bengal spice, which is a Celestial Seasoning herbal tea that is caffeine free. I went caffeine free a while ago, and it’s fantastic and I don’t have to put sugar or honey or anything in it. It has cinnamon sticks and it’s delicious and my best friend in the world got me hooked on it. And it always just makes me feel warm inside and makes me think of the lovely conversations I have with the people who I’ve drank tea with in the past. And I also have iced tea because I continuously drink iced tea and this one happens to be actually just a Crystal Light [LAUGHTER] iced tea as well.

Rebecca: And I did notice, of course, and must state that there was a Tea for Teaching mug involved.

Lillian: Yes, exactly, and I appreciate that. John saw me at a conference and I was so happy to get my Tea for teaching mug that I made sure I was drinking from it today.

John: And I am drinking a peppermint spearmint blend today… also caffeine free today.

Rebecca: I am drinking a highly caffeinated afternoon tea [LAUGHTER] to make up for everybody’s caffeine deficits.

Lillian: Well, I have plenty of chocolate throughout the day that is not caffeine free.

John: So we’ve invited you here today to discuss your work on Universal Design for Learning. We probably don’t have many listeners who aren’t familiar with UDL, but for those who are not as familiar, could you provide an overview of Universal Design for Learning.

Lillian: Of course, and I’d be glad to. Universal Design for Learning is a way of thinking, I would say, about teaching and learning that relies on three main concepts that include: engaging your students, providing accessible materials for your students, and varying the ways that your students can explain to you that they’ve learned something. And it is all based on neuroscience and also a lot of research that tells us that all of our students are different. Variability is the norm. And so we have students in higher ed that come from all different backgrounds, different cultures, different preparedness levels, different abilities and disabilities. And in order to reach all of our students, it’s important to think about that variability and universal design for learning gives us some really specific things to look out for. And three areas or categories: the engagement, the representation, and providing multiple means for those and multiple means of the student to express their knowledge. And so that’s like the general overview, but there are like so many weeds, that I can have a podcast [LAUGHTER] and talk to people about all of the different intricacies of UDL. But in general, that’s it in a nutshell.

Rebecca: I know we’ve talked about it a bit on our podcast before in terms of the difference between UDL and accessibility. But for those who haven’t heard those previous episodes, can you talk a little bit about the difference between those two ideas?

Lillian: Yes. So, accessibility is a part of Universal Design for Learning. You cannot have universal design for learning without attention to accessibility. But accessibility alone is not universal design for learning, it’s a part of it. And I like to think of UDL as like a three-legged stool. If you take out one of those legs, the stool doesn’t function anymore, it will fall over. And one of those legs is multiple means of representation. It’s typically the center column that’s purple if you look at the UDL guidelines at UDLguidelines.org. CAST puts those out. And that middle column about representation is about providing multiple forms of representation, so different texts or audio files, or making sure your font is readable too, especially for dyslexic students. So accessibility is about making sure that all students can access materials, that they have the ability to make sure they can actually get at that information. And this is less common now, but there used to be like those PDFs that a professor copied out of their book when they were a student and it has all their like markings and stuff in it. And then they would just like make a read really bad copy of it, and hand it out when everybody still used paper all the time, or maybe it’s on a screen. And those oftentimes were not accessible. And that means if a student needed a screen reader to read it, or even if they just needed to make the font bigger, they couldn’t because it wasn’t an accessible document, a screen reader couldn’t read it, meaning if a student was dyslexic or even blind, they had really no way to get at that information. So when you make your documents accessible, and you make your class accessible, you are making sure that everybody really just is on an even playing field, and they can get at that information. But in addition to accessibility, you want to make sure you’re also giving lots of opportunities for students to express what they know in different ways. And you’re also engaging with the students. And those two other things are not included in accessibility. That’s what makes up UDL.

John: How did you become interested in UDL?

Lillian: Well, I started working with our Center for Teaching and Learning in 2016 as a faculty fellow and I started doing faculty development as a one-course release for quite a while. And in doing that, we became part of a grant that was in three North Carolina institutions that was called College STAR. And STAR stands for supporting transition, access, and retention. And it was a two-pronged approach where there was a lot of student development, so tutoring centers and things like that. And then there was this other side, and that’s the part I got interested in and got pulled along with and that was supporting faculty. And to do that we used Universal Design for Learning. And I was just part of that grant. And that ended up me being the Universal Design for Learning Coordinator at Appalachian State. And I started going to different departments and introducing that at workshops and that sort of thing. So I became kind of the UDL girl or UDL lady for App state, which I think is about 2016 when that started, and then I saw it, like I’d never heard of it before, and I’d been teaching since 1997. And then I thought, oh, boy, this is really good, this makes a lot of sense. And so I started implementing it as well in all my classes.

Rebecca: So we’ve talked a little bit about UDL principles broadly. So can we dig into maybe a specific example, like your first-year seminar course? That sounds pretty interesting.

Lillian: Absolutely. So my course right now, I teach one called intercultural dialogues, and I get first-year students, and so I love it that they’re small, under 24 students. This past year, I was at our new campus in Hickory, and I only had 18 students there. And we get to work on intercultural competence, which is one of those major things that colleges want our students to know. It’s a 21st-century skill, and it is about understanding our own cultures, and then understanding that other people have different cultures. And then how can you mindfully act and interact with somebody from a different culture. And I have heard some students, at the end of the term, say “this is something everybody should know, like, this is really, really important.” They’ve seen how important it is. But at the beginning of the class, like nobody wants to be there. Nobody wants to take a first-year seminar course. They’re usually there because it’s Tuesday morning and they have an open slot. I’m in that unenviable position of teaching first-year seminar, and it’s that Gen Ed requirement that nobody wants to take. So I want to make it interesting, and I want to make it worthwhile for them, and so we learn about our own culture and then I also match students up with students abroad. We’ve worked with students in Doha, Qatar, in China, in Morocco, in Germany, Thailand, and Japan in the past year, and they work on intercultural competency skills and talk about things like power distance in the classroom, or some of the UN Sustainability Development Goals, like gender equality by 2030. And so they’ve done some things with students in China about that. So the class itself is about intercultural competence. And I have infused a bunch of UDL into this class over the years. And so it wasn’t all at once, but it took a while. And so that means I have multiple ways for the students to get at that information. So there is never just one way to do the reading. It’s either accessible or I’ve recorded a voice audio file for students. So we have a lot of commuter students at this campus, and so actually, we have zero students who live on campus in our new campus in Hickory. It is a new campus, it just started and so there are no dorms, there’s only one building and we’re all in that one building. And so students come in, they have jobs, and so UDL is very helpful for me to think about those students who have various commitments of their time. So they could listen to the audio file rather than read the book, because they’ve got a 40-minute commute to come into campus or something like that. So they’ll have an audio or video. I’ve used H5P, which is on our learning management system, which is like an interactive video, there’s VoiceThread, which is another way to be kind of interactive for students to participate. And so there’s always multiple ways for them to get at that information. And then there’s multiple ways that I ask them to tell me what they know. And they’ve done concept maps, so we have very little like, “write me a paper,” there’s very little of that my class. And that’s also a culturally competent type of teaching thing, because we often in an individuated, Western academic model, we prioritize reading and writing, reading and writing, reading and writing. That’s always what it is. And yes, we need to have very good readers and writers. But there are lots of other ways to learn and express your learning that might be more prevalent in other countries and in other cultures. And so they’ve had to draw some of their answers, they’ve had to give me a visual representation, they’ve written a poem, drawn a cartoon, and tell a story. One of their first assignments is to bring in something that expresses who they are, a cultural artifact. And then the last class that we just had recently, I asked everyone to name everybody else’s cultural artifact, so they learned about each other that way. And it was things like a keychain or one student brought in the T-top of his T-top convertible car. [LAUGHTER] Because the car was really important to get him around and all that stuff. And engagement is the last part. But it’s really the first part that gets students interested in the activities and in the learning and why they should learn. So I start off with a liquid syllabus, which is a syllabus that students can access outside of our learning management system, and they can see what we’re doing, and they get a video of me talking to them. And that’s supposed to be engaging. And then they have a lot of authentic assignments working with students overseas. And this year, because I was finally back in an actual classroom and not doing remote teaching, as I have been since 2020, I took students up to New York City. And Appalachian has this amazing loft that actually anybody can go stay at so all of your listeners could go and stay at Appalachian’s loft, and it’s very inexpensive. It’s like $70 a night per bed, and it’s these two rooms of 10 beds each. And we took students up there to actually learn a lot about culture. We went to the Tenement Museum, they looked around, and it was very cool. So experiential type of education as well. So that took like years before I got to that rendition of how I teach that course. But those are all UDL principles that guided me.

John: You mentioned collaboration with students in other countries, what types of collaborative work did those students do in your classes?

Lillian: They had specific zoom meetings that they had to do personally one on one. So one of them is that students got matched one on one with fellow students in Morocco. And they were supposed to talk about the difference in power distance in an educational setting. So power distance is the amount of power that people in a group expect and believe should be shared or held by the people in that group. So in an educational setting, if you’ve got a first-year student who comes into a large lecture hall on a college campus, there is a larger power distance for that instructor who will pretty much lecture to those students, students don’t raise their hand all that often, there’s not a lot of back and forth, there’s not a lot of flexibility, they probably don’t even know the students’ names. And so that would be an example of a larger power distance. A smaller power distance might be in a classroom, like in my classroom, I say, “You can call me by my first name, you can address me in this very informal way. We’re not going to have a lecture, we’re going to be in small groups, and then we’re going to share our ideas.” And that way, there’s a lot more voices, there’s a lot more talking. And that can happen in various times throughout the semester. I may do a lecture, I may not. And so my students were talking with students in Morocco to find out about their understanding of what power distance was. And do you call your professor Dr. Smith? Or what do you address them by? Are there rules about when you can address your professor and those sorts of things. So that was one of them. And then we worked with students in China, and this one was a series of three Zoom conversations. And all of our students had to set all these up. They were all in English because our partner students wanted to do this in English, and most of them had never spoken to a native English speaker. So this was a really good goal for them. And in China, they can’t have Zoom, not allowed. And so the students had to receive our invite from our students, and their first session was kind of an introduction: who they are, what they’re doing, for about an hour. The second was a second list of questions, which was about: Who takes care of children? Who goes to work? Who do you live with? Do you live with an extended family? Do you live with a nuclear family? And it was really about gender roles. And one of those things that’s a national cultural dimension is something called achievement versus nurturance. And that continuum has also been called in the past: masculinity versus femininity. And it’s how much a culture believes that men and women should adhere to somewhat stereotypical gender roles. So are there women CEOs, and stay-at-home dads? In some countries, that happens in some cities. That happens a lot more than in other countries. Do you put more emphasis on earning a higher wage? Or on having the flexibility to work from home? Like, where are you on that? And so the students talked about that. And then in the last session, they talked about: if you could change anything, what would you see that might improve your country from where it is now and that sort of thing. So they got to do some really authentic conversations with people around the world, and the students in China were 12 hours ahead. So my students were meeting sometimes at two in the morning, but they were up, [LAUGHTER] it didn’t matter.

Rebecca: So it sounds like there was a lot of coordination with counterparts around the world to make sure that you designed experiences for both sets of students that met, maybe not the same learning objectives, but learning objectives that were relevant for each population.

Lillian: That is exactly it. And it was my colleague in China, who said she wanted to do something about the sustainable development goals from the UN. I said, “Okay, well, let’s try and look at that.” And it worked for each one. And it is a lot of coordination for the faculty. And so I would meet with my fellow faculty member several times throughout the semester. And so we got the dates right for when we’d have Thanksgiving, nobody else had Thanksgiving break, and we have holidays, and they started a month early or a month late, and so there was a lot of coordination. And then they had to give me the list of all the students and I needed gender, too, because some students wanted to stay within their own gender, women, especially, in Morocco where some were less likely to speak to male students, who wanted to stay with female students. So we wanted to be culturally sensitive to those types of things. So there was a lot of beginning coordination to set those things up.

Rebecca: I wanted to circle back to one other thing you said too, in that you mentioned your classes developed with all these UDL principles over a significant period of time.

LILIIAN: Yes.

Rebecca: I want to know how you got started, what was the first thing you implemented? And how did that set a trajectory for the others?

Lillian: The first thing, way back when, probably 2016, 2017, like the big aha moment for me, was not doing the same thing all the time, and not having to grade everything, meaning maybe we were just going to do some honestly experimental assessments in class that were kind of fun and authentic. And I didn’t have to grade everything. And when I kind of let go of that, it opened me up to some more ideas. And then I thought, well, I don’t need them to write a paper, because I really have a specific goal in mind. And the goal doesn’t necessarily need to be a paper that then I’d have to read, [LAUGHTER] it made my life easier too. Maybe they just needed to demonstrate their understanding of these concepts. And so like one of the first things we do is draw an iceberg, and talk about how the culture that we see here, taste, feel, smell, all those five senses, that’s about 10% of what makes a culture. And when we think about culture, it’s usually just those things, it’s like, “Oh, you eat this special meal on Lunar New Year, and you have these special foods, and the kitchen always smells this way, or we dress up in cultural clothing, or whatever.” But that’s really only 10%, the tip of the iceberg. And then we have to get really deep into what our values and beliefs and assumptions we make. And that’s typically the hard part of the class. And so I just had students either draw an onion or an iceberg. And then they had to kind of point to where this was, what are your deeply held beliefs and assumptions, and that culture is so much more. And it’s a lot easier, I think, to conceptualize it as a drawing than it is to write me a paper about what your deeply held beliefs are, [LAUGHTER] and where they align with the things that I can see on the outside.

John: You mentioned that the first thing you had done was reducing grading and doing more formative assessments, which is beneficial for students too because it takes some of the pressure off and gives them the opportunity to try something, make mistakes, and learn from that without any penalty. Is that something that you’d recommend for someone who’s interested in exploring UDL, as a first step, if they’re not already doing that?

Lillian: Absolutely, I think it frees up both the student and the faculty member to kind of see what works. And so much of Universal Design for Learning is about feedback, feedback from the students. And that is a major portion of UDL. And I should have said that at the very beginning, that you really want to be figuring out what works for the students and what works for you. So I do think that’s a great way to think about it. And also, the flip side of that coin, to me as well, was whoever is doing the work is doing the learning. So if you are always lecturing to your students, it’s hard, like, you got to put together this great lecture, I always felt like I had a top hat and a cane, you know, walking into my lecture, and yadda-dat-dah, I’m gonna, like dazzle you with my art historical knowledge, and make it interesting. And I was doing a lot of work to do that. And I have slowly moved into kind of the other end of this continuum, from lecturer into facilitator. And if I can facilitate the students working together, or a lot of feedback back and forth with me or with each other, then they’re actually risking some things like “talk to your neighbor about this,” and they don’t have to raise their hand in front of the large class, they’re actually trying and risking and doing these smaller things. And that’s where I see the learning happening. If they’re just listening, that’s fine, that’s great too, but the more they can participate in their own learning, the better it is, that I’ve seen, certainly in my classes, the more they can do, the more they’re learning. But it doesn’t mean I have to evaluate every single thing that they hand in, or that they produce. And I can certainly, on the spot, kind of tweak things and say, “Okay, let’s turn it into this direction,” or something like that. But it was the: “I don’t have to grade everything they do” and “The person who’s doing the work is doing the learning.” And it was like freedom for me to try all of these things that were totally not what I had done as a student, or had valued as a student or an instructor because I was very much in that: “Alright, you’ve got a 15-page research paper, a midterm, and a final, and that’s the art history course.” And I don’t do that anymore.

Rebecca: Well, it sounds like not only is there a benefit to the faculty, in terms of workload, joy, [LAUGHTER] etc, but also an emphasis on self efficacy for students and building confidence.

Lillian: Oh, yeah. Exactly. And they’re trying things out, and they’re seeing what works. And that feedback is really important, a big part of UDL.

John: And you mentioned that it took you time to build to where your courses currently are. Is that an approach you’d also recommend to faculty who are beginning to introduce UDL principles, because it can be a lot of work completely redesigning or transforming your teaching?

Lillian: Absolutely. I don’t know if I could have done this stuff early on in my career, because I was worried about how I looked and was perceived. I was very young, and so I think I needed to feel like I was in charge. And that power sharing was too difficult for me as a young instructor. So I understand that. And now I feel much more comfortable in the classroom. And I feel that being a facilitator is really helpful for the students. And sometimes they just want to sit and listen, but that happens too. But it took a long time to get there. And the course has evolved over a long time. And you try new things. Tom Tobin and Kristen Behling talk about the plus one mentality, just trying one new thing. And that’s what happened, is when I started this course, we weren’t speaking with students in other countries. That just sort of happened when I went to a conference and made some friends in other countries and said, “Oh, I bet this would make a lot of sense to add this in.” It’d be really authentic, which is one of the engagement principles is having really authentic learning experiences. And I used to be like, “Oh, you’ve got to plan everything out, and it has to be perfect.” And now I see that I fumble through a lot of things. And every once in a while something sticks, and it’s good. It’s a practice. They say being a doctor is more of a practice. I think being a teacher is very much a practice to see what works and what worked in my class five years ago, doesn’t necessarily work now. Things that were really fun and hot at some point, you know, like making memes is pretty fun right now, but we didn’t do that 10 years ago, and we probably won’t do it in another five years, like, what did you learn? Let’s make a meme out of it. It’s evolving.

Rebecca: It’s interesting that we’re talking about a course about culture. And you’re describing how the culture of higher ed or institutions or our classrooms also evolve. And that evolution requires risk both on the part of the instructor as well as on the part of the students, and that the UDL principles are really allowing that risk to happen on both ends.

Lillian: Yes, and sometimes there are forces outside of our control that make that more difficult, and it’s not an enviable position. So things like I wish I didn’t have to grade, but we still have to have grades in the end. And so how does that fit into your course. And so I know a lot of folks are using ungrading. I know you’ve talked with Susan Blum and Josh Eyler about various different kinds of grading, which I think is like a later on kind of thing for UDL. Start with accessibility, make sure your stuff is readable and devourable by all of your students, and then start kind of playing around with it. And then maybe I think that ungrading or different kinds of grading structures might be the last step on that process. But to each their own.

John: One of the issues involving student variability is that some students might be resistant to some of the approaches that you’re using, because there have been a number of studies that show that students often prefer to be lectured at. And it seems like they’re learning more that way, despite the evidence that that’s less effective. How do you persuade students to be open to trying new approaches to learning?

Lillian: Great question, I know exactly what you’re talking about, like students, they’re like, “this is how we learn best,” and then you actually poll their knowledge. And students in an active learning situation who kind of hated it are much more knowledgeable than the students that were just in a lecture where they really liked it, because that was kind of safe. And in the last year or so I’ve heard myself saying this a lot when I do speaking, and when I’m talking to folks, is I think everybody needs to be uncomfortable in the classroom, some of the time. We don’t want the same students to be uncomfortable all of the time. So that means varying those different ways that we assess students, so it’s not the “alright, every week, you’re writing a paper,” oh, that also gets boring. But for your great writers, it’s fantastic. But are we really finding out what that student knows? Are we finding out that they’re a good writer. And so maybe that’s a poem, or maybe it’s a concept map, or there are other ways to assess that info. And so I tell my students, like, I’m really conscious about that, like, you probably aren’t going to like some of these things, but your neighbor isn’t going to like the next thing. And so having those opportunities that you have to step out of your comfort zone, to get into the learning zone, but not all the way out to that outer edge of the target, which is the panic zone. And that’s actually an intercultural competence idea that I learned in that field. When you study abroad, the only way you’re really learning is if you’re in that learning mode, like if you go to Germany, but you’re living with a bunch of Americans, and you never speak German, and you go to McDonald’s, and you’re at an English speaking school, then have you really learned much about German culture? So you should go outside of that comfort zone. Maybe you’re living with a local family, and you have to speak German, but you don’t want to go and you are in like a chaotic household and they don’t speak English, and they haven’t made sure that you have any food, and you don’t feel safe, and all the classes you are way ahead of you in your German speaking. And so you’re not learning much either, you’re kind of panicking. So it’s that learning zone we have to be in and so I think in our classrooms, we need to do that too, have multiple different ways for students to express what they know, which is one of our UDL guidelines. And I am very overt when I tell students that. And I found that with student evaluations, like I would get student evaluations where they asked like, “Did you practice critical thinking skills?” And they’d be like, “No, like, I totally didn’t at all.” [LAUGHTER] And then the next year… and I think I learned this at like an academic conference… the next year, throughout the course, I’d be like, Okay, we’re gonna do this critical thinking exercise, this thing that we’re doing right now, this is about critical thinking, you’re going to use your critical thinking skills because this thing that we’ve done, that’s a critical thinking skill. Guess what? The evaluations… way up. [LAUGHTER] Exactly like you’re pulling back the curtain and you’re saying, like, here’s actually why we’re doing it, and this is what you’re doing. And so I think explaining that is really helpful. And then the students know why they need to do something like what I’ve asked them to do, why am I writing a poem or why am I drawing an iceberg? And I think we do need to tell students that and not just have them guessing, because then they’re going to be in the panic zone and not learn so much.

Rebecca: That’s a really good point to remind students that being uncomfortable and taking risks is actually part of the learning process. Can’t remind them too much.

Lillian: Yeah, exactly. It’s necessary. And they do want to just sit and doodle. And not that doodling is bad. But they just want to sit and listen and have us do all of the work. But it’s like, I know this stuff, so why do I need to explain it? You could just watch a video of me talking. We need to get you into grappling with this and doing the stuff that I know you don’t want to do and you don’t want to be here because it’s a first-year seminar and you’re a first-year student. So my heart is with all of those folks that teach Gen Ed [LAUGHTER] to the students who don’t really want to be there.

Rebecca: Switching gears a bit now, can you talk a little bit about how you started the ThinkUDL podcast?

Lillian: Yes, it’s going back to that College STAR grant. We were getting into doing like workshops, and so I was working with other universities in North Carolina. And we had a PI, the head of it was at our East Carolina school. And I said, “Do we have any multiple ways to do this. They were doing research and then some workshops. And it just made sense to me like, we need a podcast, like this would be so off brand for UDL not to have multiple means for us to get this information out. And a podcast has the added bonus of being asynchronous, so people can listen to it whenever they want. I’ve always had transcripts, too. So if you don’t want to listen to my voice, which is totally fine, you could read the transcript and you can get that information, you can see the resources. So there are multiple ways to get it. But there was money from that original grant that sent me and my shout out to Tanner, who was my first sound engineer, and we went to a podcast convention in 2018 in the summer in Philadelphia. And seriously, I didn’t know a thing at all. I didn’t even listen to podcasts then. The only one I’d heard of was the Teaching in Higher Ed with Bonni Stachowiak, and then like, “Okay, we’re gonna try it.” And the very early episodes are, I think, awful. But luckily, Tanner kind of cleaned them up. But there was a Chris Farley on Saturday, live long, long time ago, so I’m showing my age here. But he would interview, in these sketches. He’d interview people that were amazing, like Paul McCartney from The Beatles, right? And he would just fumble the whole time. Like, “Wow, so you were in the Beatles? Wow. Yeah. That’s great. So can you tell me like, what’s it like being a Beatle?” And that’s what I felt like the whole time, [LAUGHTER] like “Wow. Okay. All right.” So it took a while. But the grant helped it and for about three or four years it was grant funded. And now I’ve turned it into its own nonprofit. And Texthelp is now a sponsor. And so they do the editing for me because the grant ended. And Tanner, he was part of that grant. So i had to kind of move on. That’s how it started.

John: So you’ve been doing this for a while now with the podcast, and we’ve been listeners since the very beginning. What do you enjoy most about podcasting?

Lillian: Well, ours came out around the same time. So the nice Tea for Teaching, right? It’s like 2018. And so I enjoy talking to people. If you’re still doing it, you have to enjoy talking to people. But that’s the best thing. I talk to people just all around the world because I do want it to have a worldwide focus. And so I have listeners, the top five are in the US, Canada, Australia, Ireland, and the UK, and so English speaking countries, yay. But you can see like, all over the world on six continents that people are listening, and I had a listener in Australia say, you know, I was walking on the beach, near my home, on the coast of Australia. And that just blows my mind that people are actually listening. But mostly it’s like, I get to just learn about new ideas all the time. And you would think UDL like it’s so focused on UDL, like, there would not be enough, like I should be done with this, but there’s so much.There are 31 of these checkpoints in Universal Design for Learning, and as you mentioned, it’s really overwhelming. Like if you were to go and just look at the guidelines, it’s like a whole bunch. And it’s like, how am I going to do that? You can’t, you can’t do it all. You can’t just redesign your course right away. And so there’s just all these little conversations I can have to help people understand what you can do. And then I get to talk to really interesting, witty, awesome, brilliant people all over the world. And that’s the best part. If I could just do that, like that was just my job, I would love that.

Rebecca: Definitely something that John and I enjoy too. It’s kind of an introvert’s dream to talk to [LAUGHTER] a lot of individuals one on one rather than having to network through a conference or something like that. It’s a good opportunity to have really in-depth conversations with folks that might not have the opportunity to have otherwise.

Lillian: Yeah, my brain is always seeking out the new. And so I love like, “Oh, that’s a neat idea.” And then I’ll send them an email, and sometimes they write back, and “oh, I really love to talk about this, it’s cool.” And so I’ll read their article or their book or whatever. And then there’s something else shiny that I get to go talk to other people about. And it’s just been helpful for folks. And honestly, I just didn’t expect there’d be listeners, and there are listeners. And so that’s just really fantastic.

John: We started out as primarily to meet our campus audience needs for commuting faculty, and so forth. And then we were amazed at how it caught on and spread. And it’s given us that opportunity that you both mentioned, to talk to some really interesting people doing some really interesting work. Before that in the teaching center, we talked to people at a workshop, and we might hear from one faculty member for three or four minutes, maybe 10 or maybe they’d come in for a consultation. But usually that was about a problem or an issue they were facing, but it just provides a wonderful chance to connect to people that we normally wouldn’t be able to talk to. And we see an interesting article, and then reading through it and getting to talk to the people doing the research in depth, it’s really a valuable experience.

Lillian: Yeah, everybody should be a podcaster just to have these conversations. You don’t even have to record them. It’s just really neat. And so it’s given me that like, “Hey, I have a podcast,” like a reason for me to be intrusive in somebody’s email. Like, I really want to talk to you about this. This is really cool. Would you talk to little old me? If so I have a podcast. [LAUGHTER]

John: And I’m amazed at the number of people who say yes.

Lillian: Absolutely, me too, like, “Wow, you’re actually going to talk to me. That’s so fantastic. I appreciate that.”

Rebecca: We’re definitely a part of a really wonderful community of practitioners.

Lillian: Yeah, it makes a very thankful and it’s so cool, because I have listened to Tea for Teaching for a long time. That’s actually my most listened to podcast for teaching and learning. I enjoy the fact that there’s two of you, and you kind of go back and forth and just interesting topics. So I’ve enjoyed yours, ever since the birth of our podcasts in 2018. They’re siblings. [LAUGHTER]

Rebecca: They are.

John: So we always end with the question. What’s next?

Lillian: I am so excited about what Appalachian State is doing. About two years ago, they bought a building in a town called Hickory, North Carolina. And our main campus is in Boone. And it’s a beautiful campus up in the mountains. And I happen to live halfway between these two cities. And so I was going up to Boone to teach and it is about 42 minutes to get up. And you have to go over the eastern Continental Divide, over the Appalachian Trail, over the Blue Ridge Parkway in order to get from my house into Boone to teach. And that’s great. And it’s the most beautiful commute I think in the world. But it also gets foggy and icy and weather and I was enjoying it, and it was where I would listen to podcasts. But Appalachian State is now the first university in the North Carolina system that now has two campuses. And so we’ve opened this campus in Hickory, it is a commuter campus, and some brave students, we have about 250 to 300 that have started this past fall of 2023. It has birthed this campus. And so I get to teach there, and I get to do some faculty development. And it’s really exciting to be on the ground floor of a new campus. And it’s the only one in the North Carolina system. We’ll have other campuses, but there’s no multi-campus university for us. It’s like being in a startup, except I don’t think I get stock options. That’s the only bad thing. [LAUGHTER] And so meeting new faculty, some faculty are teaching for the very first time. And so there’s no like institutional culture that they’re jumping into at this new campus, although we are very much a part of the Boone campus. It’s new. And there’s only a very small number of faculty there. So it’s like being at a small liberal arts college in a state system. And it’s just really cool. And I’m loving meeting the faculty there and helping with teaching and learning and UDL and all that stuff. So that’s like the next big thing is Appstate Hickory, and it’s really exciting.

Rebecca: Well, I hope you have a wonderful adventure. [LAUGHTER] It sounds like a really fun opportunity.

Lillian: Yeah, in fact, I’m in a group of faculty, we have like a community of practice, a peer mentoring circle we call it, and we’re calling ourselves the Hickory Adventurers, because like, we don’t know what’s going on, [LAUGHTER] and we’re trying to figure it out together.

John: You get to help shape what’s going on, which is a really nice place to be.

Lillian: Yeah, it’s fantastic. I’m excited. And I think I’m just that kind of person. It’s new and shiny. And I’m there.

John: I think we’re both that way a bit. And that is one of the risks of having a podcast, you get to hear about all these great things that people are doing, and there’s always a tendency to try to do many of them. And that can be a bit overwhelming, not just for us, but also for our students.

Lillian: Yes I know I have to peel it back [LAUGHTER] just a bit, don’t go overboard.

Rebecca: Well, thanks so much for joining us, Lillian. I know we’ve wanted to talk to you for a while.

Lillian: Absolutely. I’m so glad and when you contacted me, I was super excited. So thank you so much for having me on Tea for Teaching. I’m gonna show my mug that nobody can see ‘cause it’s a podcast, but I love my Tea for Teaching mug, and thank you for having me.

John: Well, thank you for joining us. It was great talking to you and we’ll look forward to more conversations in the future.

Lillian: Great.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

324. Unmaking the Grade

A growing number of faculty have been experimenting with ungrading. In this episode, Emily Pitts Donahoe joins us to discuss her ungrading approach and the documentation of this process on her blog. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi.

Show Notes

Transcript

John: A growing number of faculty have been experimenting with ungrading. In this episode, we discuss one instructor’s ungrading approach and her documentation of the process.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Emily Pitts Donahoe. Emily is the Associate Director of Instructional Support at the Center for Excellence in Teaching and Learning and Lecturer in the Department of Writing and Rhetoric at the University of Mississippi. She experimented with ungrading and chronicled her experiences in her Unmaking the Grade blog. Welcome, Emily.

Emily: Thanks. I’m really excited to be here.

John: We’re very pleased to be talking to you. We talked recently at the POD conference, and I’m looking forward to this additional conversation.

Emily: Yes, where I got this lovely tea for teaching mug from John, which I’m so excited about and drinking from right now.

John: And since this is only audio, Emily was holding the mug [LAUGHTER]…

Emily: I’m showing it off.

John: So our teas today are:… Emily, are you drinking tea in that mug?

Emily: I am. [LAUGHTER] I am drinking tea out of my tea for teaching mug. I’m a big tea drinker. And so today I’m drinking my favorite tea, which is a strong black tea called Scottish morn. And I got this tea from a tea shop called Apothica in Niles, Michigan, which I used to go to when I lived in South Bend, so highly recommended if you’re in that area.

Rebecca: Sounds wonderful. I was ready for you to say that you had coffee or something in the tea for teaching mug so it’d be completely blasphemous… [LAUGHTER]

Emily: Never.

Rebecca: …because coffee is one of the most frequent flavors. [LAUGHTER] Emily, you’re doing it right. [LAUGHTER] I have blue sapphire tea today.

John: And I have Irish Breakfast tea today.

Rebecca: So, we invited you here today, Emily, to discuss your experiences with ungrading. Your blog is based on your spring 2023 course experiences but your ungrading experience predates this course. Can you tell us a little bit about your initial experience with ungrading?

Emily: Sure. So I have a background in Writing and Rhetoric and English literature, and so a lot of what we do in Writing and Rhetoric, I think, is already pretty aligned with some ungrading goals and practices. So in my courses, students have always had the opportunity to revise their work based on feedback, and include it in a final portfolio, and most of their grade is based on their revised work. So it’s not just that they get graded and then that’s the end of it, they always have a chance to improve based on feedback. The first time that I stopped putting letter grades and percentages on student work was in spring of 2022, when I was teaching a general education literature course at Notre Dame, and so the course was on pre-modern and early modern literature, but with a focus on how the texts that we were studying might help us examine or wrangle with some of the questions that we’re preoccupied with as a culture and society today, and thinking about how those texts might relate to student lives. So it was a course for kind of all levels and all majors, which I think made it a good course to experiment with. So I had first-year students all the way up through senior-level students from engineering and business and English and psychology and all kinds of different majors. So like all teaching experiments, I think there were definitely some kinks to work out after that first course; they never go exactly right the first time. [LAUGHTER] But I would say that, overall, it was a huge success. And that’s not because I did it perfectly, or even because I did it particularly well, but because I think ungrading really helped some of my students move beyond this idea of the school as a points game, to help find their interest and their motivations to study the material that we were studying for their own purposes, and to focus on developing the skills and knowledge that they wanted to develop rather than on attaining a specific grade.

John: And the second time you did this, you created a blog describing the process. What prompted you to create the blog?

Emily: So, I was inspired by a post written by Robert Talbert, who’s the co-author of a recent book, Grading for Growth, which I really highly recommend on alternative grading, and he and David Clark, his co author, also had a blog and substack newsletter on Grading for Growth. And so in December of 2022, Robert posted a stop, start, continue for the ungrading community. And if you’re not familiar with stop, start, continue, it’s an evaluation exercise used for mid-semester reflection, very often in classrooms where the instructor or another facilitator will ask students: “What kinds of teaching practices or classroom practices would you like to stop, start, or continue in the class?” And so Robert’s post was a stop, start, continue for the ungrading community. And one of the things that he recommended that the ungrading community start doing was getting into the weeds and writing in detail about the daily experiences and specifics of upgrading, so: what we’re doing, what kind of successes we’re having, what challenges we’re encountering, how we’re adjusting in real time, and he recommended keeping this as the kind of blog or like a captain’s log of weekly reflections. So when I read his post, I thought, “Well, I could do that. It didn’t sound that hard, and it sounded like a lot of fun.” So that was what prompted me to start writing weekly reflections to share some of the methods that I was using, successes and challenges that I had, and even some of my doubts or misgivings about some of the things that I was doing in the class.

John: And just as an aside, we were so impressed by Grading for Growth, that we’ll be using it in the spring reading group here, both at SUNY Oswego and at Plattsburgh with Jessmyn Neuhaus. So we’re very much looking forward to that. And Robert and David will be giving a keynote address at the start of our workshop series in just a few weeks in early January. So we’re very interested in doing more of this on our campus as well.

Rebecca: Can you talk a little bit about your decision to delay posting the blog until after the semester ended?

Emily: Sure. So the selfish reason is that it’s supposed to be an unfiltered look at the ungraded classroom. And part of me thought, what if I messed this up so badly that I’ll be embarrassed to talk about it. So it was partly that, but I would say the more important reason is that I thought it might be weird for students if I was talking in public about what was happening in the classroom in real time. And I was concerned that it would damage the relationship of trust that I wanted to build with my students if they knew that I was reporting on our conversations to a larger audience about the class. So I offered to hold things for that reason. But the other thing that I wanted to do was make it clear to students that I was writing these reflections that might later be published, and then to get their input on them. So some of what I shared on the blog, I didn’t really want to share without getting students consent to do that. But I also wanted to get their input on some of the things that I was talking about. So I believe really strongly in taking a students as partners approach to higher education and learning experiences. And I try to employ that in my educational development work in the classroom. So that means bringing students into conversations about teaching and learning and asking for their experiences and their expertise as students really and then using that to inform our practice. So delaying the posting allowed me also to get some input from students and to be able to share some of their thoughts and opinions on the blog whenever I could.

John: In reading through your blog, your spring 2023 class seemed really interesting. Could you describe that for our listeners?

Emily: Yeah. So this is the second course in our sequence of first-year writing courses here at the University of Mississippi, but it’s a little bit different. So for the second course in the sequence, students have the opportunity to take either Writing 102 or Liberal Arts 102, and I was teaching Liberal Arts 102. And so the goals of each course are really the same. But Liberal 102, as we call it, is conducted within the context of a research area in a specific discipline. So that means that it can have a very specific theme. So some of my really excellent colleagues in the department and in other departments have taught courses like writing about true crime, or the rhetoric of sports, or I believe there’s a course on fashion. So because one of my areas of expertise is teaching and learning, the course that I taught was called Examining Higher Ed Teaching and Learning in a College Classroom. And so I think this is a great course for first-year students as they’re entering this next phase of their education to reflect on where they’ve been and where they want to go. And so what we did was looked at questions like: What’s the purpose of a college education? Why are we all here? What kind of benefits does it provide to individuals or to society? What kind of collective benefits does it provide? And then how are those benefits enacted or engendered in the classroom. And so we explored a lot of debates around higher ed in the US and students have an opportunity to reflect on and draw on their own experiences as students and their expertise as students, and then integrate that with larger areas of research on education or current events in education. And then they communicated their ideas about education to audiences outside of our classroom. And so it was really, I think, an ideal course for ungrading because we could talk about grades, not only as a matter of course policy, but also as a core subject matter. So in the beginning of the semester, we read Alfie Kohn’s piece “The Case Against Grades” and talked about it both as a way to introduce the course grading system and as a kind of larger prompt for reflection about grades as an issue of concern in higher education right now.

John: And it sounds like a great opportunity to have students reflect on what you’re doing as you’re doing it. How did students respond to this?

Emily: So I think every time I’ve ungraded a class now, and I’m currently on my third ungraded class, students have responded a little bit differently every time and of course, every individual student is also different. I think there are three broad categories of response that I see. One is just enthusiasm; some students are really excited about ungrading, and usually that’s students who feel that, for one reason or another, their grades in the past haven’t been representative of their learning or who feel that some of their creativity or their risk taking has been stifled by their desire to get a good grade. So I wouldn’t say this is a lot of students, this is probably a smaller group of students, but some students are really excited about it. I would say another group of students are really hesitant because ungrading is a big unknown for them, especially for students who are dead set on getting that “A” grade. It can be really nerve racking not to know kind of where you’re at in the middle of the semester. When I get negative feedback on ungrading, it’s almost always students who say,”This is really interesting, but I never know where I’m at.” And so they’re really concerned about how they’re going to measure their progress. But they’re not thinking about their progress in learning, they’re thinking about their progress toward a specific grade, which is really understandable because grades are important. And so that’s also a smaller group of students. But I would say, by far, the biggest reaction I get from students is something like cautious optimism. So I always start ungraded courses with a conversation about grades and learning, and I ask students to share with me their experiences of grades and share with each other. And very often, they have a lot of negative experiences to share, and sometimes positive experiences as well. But we talk about the relationship between grades and motivation and grades and learning and they have a chance to reflect on that. And so I use these conversations as a jumping off point to explain why I use the system that I do and how I think it will benefit them as learners. And I think students find it really helpful to be able to talk about their experiences with grading openly and to be heard by a teacher. And I think that that alone makes them more willing to buy into the system. So once it’s explained fully, and once students start to see the potential benefits, I would say that most of them are cautiously optimistic.

Rebecca: I think that largely aligns with my experiences and explorations when ungrading as well. In your class, you included five different assessments and opportunities for revision. Can you talk about and describe these assessments?

Emily: Sure. So the first thing that students had the opportunity to do were weekly writing practice assignments. So these are what you might think of as formative assessments, they’re preparation for class discussion and major assignments that mostly involve reading or short writing prompts. And these students weren’t able to revise, they either kind of did it or they didn’t. But then students had an opportunity to do more longer major assignments, which are more typical assessments for the writing class, a series of papers and a multimodal kind of project or two, with an imagined audience kind of outside our classroom. And so these students could revise up to three times if they wanted to do that. And so I had to, for this class, adjust a little bit and do slightly fewer assignments than I might have done in a traditionally graded class, because the expectation is that students would be doing more intense work on each assignment in their revision. I also asked students to do self assessments periodically throughout the semester. So students would answer a series of questions about their progress toward the learning goals, about their goals for the remainder of the semester. And they would also propose a current grade for themselves based on evidence that they provided. And this is a similar activity to what we did at the end of the semester to determine their course grades. They were also assessed on their final portfolio, which had revised versions of their major assignments. So their work… basically as good as they could make it… their best version of their major assignments, plus another final self assessment of their work in the class. And then they were also assessed on class engagement, so their class attendance, their preparation for and participation in class discussion, the timeliness of their work, their support for the learning community and fellow students and things like that.

Rebecca: Can you talk a little bit about how you provided your feedback on those assessments?

Emily: Sure. So for the writing practice assignments, usually I just wrote a sentence or two in response to students work to let them know if they were on the right track, or they may be needed to adjust slightly as they prepare for their major assignments. For the major assignments, I did do a lot of longer written feedback, and a lot of this looked like feedback that I would give in a traditionally graded class. But I think it was a little bit more oriented toward future growth, rather than reflecting on past work. It was giving them a lot of direction on how they might want to revise to make their work better in the future, not just for future papers, but for this particular paper. And with that feedback, I also gave them a rubric, where I marked kind of where I thought they were at along the specific assessment categories and criteria that we were using for that assignment. So for example, we might have a category for audience and purpose. Students could see whether I think their work on that category was still developing, whether it was at the level of proficient or whether it reached the level of excellent, so I had these different categories and these different kind of progress metrics, where I would indicate this is a skill that I think you’re still working on, or this is a skill that you’re doing really well in. And I also provided feedback to students through our individual conferences. So we met once at midterm, I met with each individual student, and then once at the end of the semester, to have a face-to-face conversation about how students were doing and if they needed to make any adjustments to reach their goals for the rest of the semester. And I didn’t give any feedback through letter grades or points. Those are not really great forms of feedback. They don’t actually give us a lot of information about our progress. So most of the feedback that I provided was kind of qualitative feedback.

Rebecca: You just talked a little bit about rubrics as part of a feedback that we were providing students, can you talk a little bit about how those rubrics were developed?

Emily: Yeah, so I’ve tried several different methods of creating rubrics. And one of the things that I like to do is co-create them with students or get students input on rubrics. And I attempted to co-create rubrics with students in two different classes and have done it a few times. And sometimes it’s a huge success, and sometimes it really does not work very well at all. And in my spring class, it didn’t work very well at all. [LAUGHTER] So basically, what I’ve done previously is ask students what they think makes a good argument or a good piece of public writing, or whatever it is that we’re working on. Or I ask them what kinds of things they believe their writing should be assessed on, and how we would determine whether or not their writing is successful. And then we have a conversation about that. I make notes on the board as we’re talking. And then I go away and distill those notes down into a rubric with about five categories in which student work is assessed. And then there’s the three metrics, so developing, proficient, and excellent. And sometimes this works really well. But in my spring class, I think the students were really at a loss when I asked them about assessment, and even when I asked them what good writing looks like. And I think that’s because that no one has ever asked them to think about that before. They’re used to being told what their work should look like by someone else, and then trying to conform their work to those expectations, or to someone else’s standards. And obviously, we need to have standards and expectations for student work. But one of the things that that does is get the students only thinking about what the teacher wants and not what they want. They’re not thinking about what they want their work to look like, or what their standards are or what they’re trying to accomplish. And so I think, ultimately, if we’re preparing students for the real world, whatever that is, this is what we want them to be able to do, not just to blindly follow somebody else’s standards, but to create and work independently in a self motivated way, and then assess their work independently. So I think we have to kind of start with baby steps in that process. So in the spring course, the rubric activity really turned into a conversation about how students have been assessed in the past and why they’ve been assessed that way. So when I ask them about their experiences, how their writing had been assessed, they would say things like, we were graded on the word count, or we were graded on whether or not we included two sources in every paragraph. And then we use that as a jumping off point to talk about the principle behind those what they viewed as arbitrary rules. And so if your teacher is concerned about word count, it’s because they want you to make a substantive argument. And what does a substantive argument look like? If your teacher is concerned about including sources, it’s because they want you to make an argument based on strong evidence. And so what would strong evidence look like? And so we use that conversation to think about the assessment criteria for their college level work, and how that might be similar or different from what they experienced in the past. So in that way, I do think that conversation was helpful. And I do think based on my few experiences, that it helps to have students extrapolate criteria from some examples, rather than from thin air, kind of as I was asking them to do, but either way, having a transparent conversation about what students are being assessed on and why and a purpose behind the criteria is key to their success.

John: In one of your blog posts, you describe some of the writing assignments that you used in your class, and they seem quite interesting. Could you describe some of the writing prompts that you gave to students?

Emily: Yes, I was really excited about some of the assignments that I developed for this class. And I think I’m going to keep using them or versions of them in future classes. So for this class, I had a menu of assignment options. So the students could, for each major assignment, choose the kinds of projects that they found most compelling, or they would really draw on your strengths. And I thought this was important, partially because we were working some with generative AI in the class. And I’m sure everyone is aware by now some very serious data privacy concerns, ethical concerns around the use of generative AI. So I didn’t want to mandate that students work with AI if they didn’t want to, and some really didn’t want to. So I tried to offer them several options. And I was really happy about some of the options that I came up with. So obviously, one of the things that I was thinking about going into this semester was how to deal with generative AI and I have a lot of thoughts about that. But one big way to think about how to discourage inappropriate use of AI or encourage appropriate use of it is to think really carefully about assignment design. So one of the things that I did to help students navigate AI was lean into some multimodal work, think about argumentation in media other than writing or that kind of worked alongside writing. So we didn’t abandon writing entirely, but I did include a photo essay as one possible assignment in a menu of options. And for this course, I asked students to take a look at a book of photographs by Cassandra Horii and Martin Springborg that’s called What Teaching Looks Like. It’s a really, really cool book with candid photos of college teaching. And so we use this book to talk about visual rhetoric and also to use the photos as a launchpad for this discussion about college teaching practices and students’ experiences in the classroom. And so the assignment that students did based on their work with this book was to create their own photo essay called “What Learning Looks Like.” So instead of what teaching looks like, what learning looks like, with a specific audience and purpose in mind. So that was one of the assignments that I liked. Another one that I thought was really promising was something called “share your story,” which asks students to tell a personal narrative about their educational experience, and then connect that to a larger issue of concern or a body of research in higher education. So of course, this is not totally AI proof. ChatGPT can make up a story, but it can’t tell my students’ own stories. And I think most of us really like to tell our stories to other people. So I think that provided a little more motivation for students to do their own work. And then the last assignment that I really liked was an assignment that actually asked students to work with ChatGPT. So they created a prompt based on a template that I provided for an argumentative piece that they then fed to ChatGPT, so they gave ChatGBT the prompt. And then they took the ChatGPT output and critiqued it. So they annotated it, noting what pieces they thought were strong, whether or not the piece had weaknesses, and what those weaknesses were, where it might need revision or overhaul. And then they had to totally rewrite the piece that ChatGPT produced and make it their own. So they had to do a substantial revision. And then they had to annotate their own work and tell me why they made the revision decisions that they did. And so I want to clarify that I do think it’s important that students learn to generate first drafts on their own, because drafting is an essential part of the writing process, and it’s where a lot of the thinking happens. But I like this assignment as just one assignment in the sequence, because it does help students learn about generative AI and develop some AI literacy. And it also helps students get over that terror of the blank page. So a lot of students procrastinate because they don’t know how to get started. They’ll open their laptop to begin a paper and stare at the blank screen. And I find it really difficult to get over that first hump of starting work, and just close their laptop and go away and try it again later, usually the night before it’s due. So I think starting in this way, with a ChatGPT prompt and an essay gives them a jumping off point, and it’s an easy way to help them start building the confidence they need to do their own first draft.

John: We should mention that we did talk to the authors of What Teaching Looks Like, and we’ll include a link to that discussion in the show notes. One of the things you described in your blog is that having this ungraded environment encourages students to be perhaps a little more open and honest with their instructors, but that could lead to some challenges in terms of additional emotional labor. Could you describe the challenges that you faced in this class with that?

Emily: I think one of the big themes in the spring class in particular was that I asked students to share with the class and for audiences beyond the class about their previous experiences in school, or their current experiences, and I think this also happens in a lot of ungraded classes. One common method of introducing an unfamiliar grading system is getting students to think about their previous experiences with grades. And so one thing that happened in this particular class is that students’ related a lot of past educational trauma to me, and usually that involves bad experiences with previous teachers. And I’m really glad that they were able to speak honestly about that sort of thing. And I think it helped them to have a teacher take them seriously when they related those stories. At the same time, it was pretty difficult to navigate those conversations, because I was managing their emotions, my emotions, and also doing that when sometimes I only knew one side of the story. And so that was a little bit difficult. Another thing is that students in this class didn’t seem to feel that their grade depended on telling me what I wanted to hear. So I think they were a lot more honest about their views than in other courses that I’ve taught. So in my traditionally graded classes, or I think, in any class where students are discussing hot button issues, they tend to think that they’ll be graded more harshly if they express views that the instructor disagrees with. So very often, I think, they try to say what they think you want to hear rather than what they really think. And that didn’t seem to happen as much in my spring course. I had several students endorse viewpoints that I definitely disagreed with whether they knew that or not. And I think that’s good, because students have to sort through their own views and values. But it also required me to do a lot of thinking about how I would approach and address students not just whose views I disagreed with, but whose expression of those views might be damaging to others or to themselves. And so, we need to do some work around community building and relationship building in the class. And then of course, when you build relationships with trust with students, they’re more likely to tell you about the personal problems that they’re facing, whether that’s their mental or physical health or their personal relationships or family emergencies or grieving. And it’s really just a lot. And I do sometimes lay awake at night worried about my students. And there’s a lot of care work happens, I think, in all classes and also especially ungraded classes. And so there’s a lot of work in referring students to other resources and helping them navigate campus resources, and also just a lot of kind of management of your own emotional state [LAUGHTER] that has to happen. So I did want to be honest on the blog about some of the emotional toll of the work of teaching in general and of ungrading too.

Rebecca: You’ve described the care work, you’ve described individual conferences with students, you’ve described students’ anxiety over not knowing where they are sometimes and allowing substantial revision. Can you talk a little bit about how all of those things play into workload and how you’ve managed things?

Emily: Yes, this is a good question. And I feel like I have a very complicated answer to whether or not ungrading has increased my workload, because I get asked this question a lot. In my traditionally graded classes, giving all that feedback felt like a waste of time for a few reasons. First, because I had the sense that students weren’t really reading the feedback. So we know from research that when students receive a letter grade and also feedback on their work, they tend to see the grade and then ignore the feedback, or at least receive the feedback as a justification for a grade, rather than something that’s going to help them improve their work in the future. So I also spend a lot of time in traditionally graded classes worrying about how students would receive my feedback if they got a low grade on their paper. So how could I write feedback that would be appropriately honest, but also appropriately encouraging, so that students didn’t just see a C-minus or a D on their paper, and then give up and throw it in the trash. So I don’t really worry about those things since I’ve adopted ungrading. I provide feedback honestly, and with the mindset of a coach rather than a judge. And I provide it with at least some confidence that students will read it and use it. So they have plenty of opportunity to revise. And in fact, their ultimate achievement of the course is measured by their growth in specific areas and their demonstration of learning that arises from taking a piece of writing from not so good to much better. And so the expectation is that what they turn in the first time is not their best work and they’ll only get to their best work after they incorporate feedback. So I do spend a lot of time responding to student work now that I’m ungrading. But the process is more efficient because it accomplishes my goals rather than wasting my time. And it’s more enjoyable, because it causes me less angst. So I guess it is more work to provide feedback, but it’s also more efficient and enjoyable work. I think I feel kind of the same way about individual conferences with students, that it does take quite a bit of time to do those and it would be, I want to acknowledge, so much more difficult, it may be impossible to do if I was teaching more classes and more students. I’m very lucky that I teach small class sizes. And because I work in educational developments and work in a teaching center for most of the time, I only teach one course at a time. So I think there are ways to do this with larger courses, but I’m very fortunate in my course to be able to conference individually with each student twice in the semester. And those conferences are incredibly time consuming, and they can be really draining, but they are also really joyful. And I think it’s really important that students are able to have those one-on-one conversations with me. And they are much more, I think, effective in accomplishing a lot of the goals that I have for student learning than just simply doing written feedback or peer review or things like that. Having that face-to-face time to give students some individual attention is a really both enjoyable and effective part of the learning process.

John: A question that often comes up from people who have not tried ungrading is how well do students’ perceptions of their learning align with your perceptions of the learning when you do have to assign those midterm or final grades in the class?

Emily: Yeah, this is a really good question. And I would say I’ve had to do a little bit of work on my process to make sure that our expectations are aligning well. Sometimes, there are cases where students don’t automatically start off knowing what I expect from their work and what good work looks like, even when I thought that I was clear about that. So that is an issue that I’m working on. I would say for the majority of students, they do understand what good work looks like. And when they’re asked to provide evidence for their course grade, most of the time they know what good evidence looks like and are able to demonstrate to me in really, sometimes ways that I hadn’t anticipated, that they really had learned in the class and progressed and improved their work. For those students who struggle, and I think it’s more frequent for first-year students to struggle with this than more advanced students, for students who do struggle, I think it is important to be able to show them models of student work early in the semester so that they can get a sense of what a successful assignment looks like or what’s kind of level of expectations we have for what student work counts as really excellent work. So that’s been really helpful. And I’ve also made some changes to my class this semester to help clarify for students a little bit what kind of evidence might be good evidence for specific grade proposals in the course. So if they are really shooting for an A or B in the course, what kinds of actions or behaviors or demonstrations of quality work do they need to be able to show in order to attain that grade?

Rebecca: I know one of the things that I struggled with in some of the classes where I’ve done ungrading is that sometimes the midterm conference feels like it comes too late. So I experimented, I think once with like a one-third, two-thirds, three thirds approach, but then that was so much more conferencing.

Emily: Yeah.

Rebecca: So I’m curious about your timing and how the midterm time works in the classes that you’ve been teaching.

Emily: Yeah, I think that’s where the self-assessment assignments come in. So I do think midterm is quite late for students to be getting the first level of feedback. And so I have students do a self-assessment form, I guess, a quarter way into the semester. So we start that process really early of having them look back at the work that they’ve done, and propose a grade for themselves based on that work. And when they do those self assessments, I don’t conference with students every time they do a self assessment, but I do look at where their grade proposals are. And I’m able to say if our expectations are very, very different, or assessments of that students work is very, very different, I’m able to reach out and say, “Hey, I don’t think we’re aligned here, and here’s why.” And being able to intervene early on that I think is really important.

John: We’re recording this near the end of the fall semester of 2023. And you’re currently teaching another ungraded class, could you tell us a little bit about what types of changes you made from the spring class to the fall class?

Emily: Sure, there are a few changes that I’ve made, because there were some things that I think did not go very well in the spring and I wanted to try to improve that. So what didn’t go well was that I had a real problem with attendance, there were quite a few students who struggled with their attendance, especially in the latter half of the semester, and quite a few students who struggled to submit their work on time kind of throughout the semester. And that made it difficult for me, but I think more importantly, it made it really difficult for the students who once they missed an assignment, they found it really difficult to get back on track. So my challenge is really to figure out how to motivate students to attend class regularly and submit work on time without penalizing them for absences or late work or without a kind of point system to encourage them to show up to class and to submit their work on time. So that was one thing I wanted to address. And then the other thing is that students’ anxiety about not knowing where they stand in the midst of the semester at any given point. So what I developed to address all of those challenges was a course progress tracker. And so this is a document, a pretty comprehensive document, inspired by David Clark’s Grading for Growth post about grade trackers that I read during the summer while I was designing this course. And so the document is really a series of tracking worksheets in three different categories. So the first category is readings and assignments, students can see each week at a glance what work they have to complete, and then they can check off boxes as they complete that work, and then note, if there are assignments they’re submitting late, they can record those late submissions. There’s a category for attendance and engagement where students can check off the classes that they attend, and then make notes about their in or out of class engagement during each week of the semester. And then there’s a section… the most important section… for learning and growth where students can remind themselves of what the course goals are, and then track their progress along those goals, so they can see and make notes about where they’re still developing, where they’re doing excellent work, and how they’re improving their writing as the semester goes along. So, so far, it’s been going really well. And I’ve had fewer challenges with attendance and late work this semester than last semester, though, I can’t say to what extent that’s just a result of a different population of students or the fact that it’s fall instead of spring. So I will add that caveat, but I have surveyed students about their use of the progress tracker last week, and I’m really looking forward to diving into that next week. So the last thing that I think the progress tracker does, which I didn’t totally intend, but which has been really excellent, is that I think it helps students a lot with their self-assessment work, which they also struggled with, to some extent last semester, or I should say, maybe I struggled to teach really well. So I had a realization at the end of last semester, that when I sat down to think about a student’s work over the course of this semester, I was really thinking about three things. And so the first thing is quantity. How much work did students do? How much labor did they put in? How many assignments did they submit at a satisfactory level? How many class days did they attend? How much time did they spend on their major assignments? So that was one category. The second was quality. So how good was the work that they were doing according to the standards that we laid out? Was it still kind of developing work? Or was it excellent work? And then the third thing was growth. So how and how much did student work improve over the course of this semester? And can they demonstrate that they’ve gained knowledge and skills that they didn’t have before? Or can they demonstrate that they’re better off from having taken the course. And so while we’ve had conversations about those things in the spring, I never articulated to myself or to my students, that particular model; I never kind of said it in quite this way. So the last section of the progress tracker includes a guide to determining final letter grades for students. And it gives them space to think about quantity, quality and growth, and how that might contribute to the grade that they propose for themselves, if they’re interested in the specific letter grade. And it gives them a sense of what kinds of evidence they could provide if they want to show good evidence for a specific letter grade. So you can provide good evidence for an A if your work rises to the level of excellent on multiple assessment categories. Or if you attended class every time that you were able and submitted the vast majority of your writing practice assignments. So all of these things are good pieces of evidence if you want to propose an A grade for yourself. So there’s some flexibility there, I don’t prescribe exactly what students have to do for a specific grade, but I do make suggestions and say, here’s a guideline for you. If you’re confused about what kinds of work you need to be doing, or what you need to do to demonstrate that you’ve attained a certain letter grade, you can take a look at this guide, and learn a little bit more about what the expectations are. Currently, it’s just something for the students to use for themselves. I have provided time in class for students to fill out sections of it, because I think that if I did not ask them to do it in class, they might struggle to keep up with it outside of class. But I think maybe in the future, I will consider asking them to keep up with it. And then periodically checking in on those progress trackers throughout the semester so that I can intervene early, if there are any issues and maybe leave comments on the documents to let students know if I think they’re struggling in a specific area, or if they’re doing really well in a specific area. I am playing around with that idea for my next version of the class.

John: So we always end with the question: “What’s next?”

Emily: So this spring, I won’t be teaching an undergraduate course, but I will be working with graduate students who are preparing to teach their own Writing and Rhetoric courses. And I’m really looking forward to talking with them about teaching. And I’ll also be continuing to write about ungrading on the blog. So specifically, I’m hoping to share a bit more about what I learned from my experiences this semester. And I’ve been collecting data from my current students about their use of the progress tracker, about their use of AI in their writing this semester, and about their feelings and impressions of ungrading. And what I’m planning to do throughout the spring is share some of those student thoughts with the readers of the blog, and I’m really excited to be able to share students’ ideas about ungrading and other topics as well.

Rebecca: Sounds great. We’ll look forward to reading that for sure.

John: Thank you, Emily. It’s great talking to you and we look forward to future conversations.

Emily: Thanks, this has been fantastic.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

315. Unessays

An unessay assignment provides students an opportunity to demonstrate their learning in innovative and creative ways. In this episode, Jessamyn Neuhaus and Maggie Schmuhl join us to discuss how they have employed unessay assignments in their courses.

Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Maggie is an Associate Professor of Criminal Justice and the Associate Director of the Center for Excellence in Learning and Teaching at SUNY-Oswego.

Show Notes

  • Neuhaus, J. (2019). Geeky Pedagogy: A guide for intellectuals, introverts, and nerds who want to be effective teachers. Morgantown, WV, USA: West Virginia University Press.
  • Neuhaus, J. (2022). Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. West Virginia University Press.
  • Neuhaus, J. (editor) (2022). Using the Unessay to Teach History. Teaching History: A Journal of Methods. Vol. 47. No. 1.
  • Cavanagh, S. R. (2023). Mind over Monsters: Supporting Youth Mental Health with Compassionate Challenge. Beacon Press.
  • Jessamyn’s American Promise Unessay Assignment
  • Pittman, Chavella (2022). “Strategizing for Success: Women Faculty of Color Navigating Teaching Inequities in Higher Ed” in Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Ed. by Jessamyn Neuhaus. West Virginia University Press.
  • Pittman, Chavella (2023). Navigating Teaching Inequities. Tea for Teaching Podcast. Episode 291. May 31.
  • POD Network Conference
  • Teaching, Engaging, and Thriving in Higher Education Series. University of Oklahoma Press.
  • American Historical Association Annual Meetings

Transcript

John: An unessay assignment provides students an opportunity to demonstrate their learning in innovative and creative ways. In this episode, we explore ways in which different faculty have employed unessay assignments in their courses

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guests today are Jessamyn Neuhaus and Maggie Schmuhl. Jessamyn is the Director of the SUNY Plattsburgh Center for Teaching Excellence and a Professor in the History Department at SUNY Plattsburgh. She is the author of Geeky Pedagogy: a Guide for Intellectuals, Introverts, and Nerds Who Want to be Effective Teachers. She is the editor of Picture a Professor: Interrupting Biases about Faculty and Increasing Student Learning. Maggie is an Associate Professor of Criminal Justice and the Associate Director of the Center for Excellence in Learning and Teaching at SUNY-Oswego. Welcome back, Jessamyn and Maggie.

Maggie: Hello.

Jessamyn: Hi. Thanks for having me.

Maggie: Good to be here.

John: Our teas today are:… Jessamyn, are you drinking any tea?

Jessamyn: I am. In preparation for this very question, I’m drinking a wild sweet orange.

Maggie: Oooh.

Rebecca: Sounds nice.

Maggie: That sounds very fall. I am drinking a bread pudding tea. It’s a black tea from a little tea shop in Sacketts Harbor.

Rebecca: That’s a first on our podcast for sure.

Maggie: I think it’s called Tea Thyme, the little shop. It’s quite lovely.

Rebecca: Nice. I have Irish breakfast this morning. So we’ve invited you both here today to discuss unessay assignments. Could you tell us a bit about what an essay is?

Maggie: I guess I’ll say an unessay is anything but an essay. Jessamyn probably has some more nuance? [LAUGHTER]

Jessamyn: Not really, that is the basic premise. In classes where you might traditionally ask students to show their learning via a standard essay, either like the five-paragraph essay or a research paper, an unessay project asks students to demonstrate their learning, increased understanding, via any other kind of format.

Rebecca: Jessamyn, can you tell us a little bit about why you decided to use unessays in your class?

Jessamyn: Sure. And this also relates to… I know, we were maybe going to talk about what was appealing for historians, in particular, about the unessay. And some points for history professors generally and me, what attracted us to considering using the unessay. A couple of things about our discipline: first, that the unessay, it’s a good way for students to explore some key historical thinking skills in different new ways for them, so things like citation, research, and historiography. So trying to understand what other historians have written and argued about the past. There’s also been a pretty significant sort of public history turn in the world at large, like trying to better understand different audiences for historical research. Some other more general reasons were issues of equity and inclusion, especially around Universal Design for Learning, trying to offer students some options as a way to increase their learning, increase engagement. And the final thing I’ll say, just for me personally, is that I wanted to make assessment and grading a more joyful and happy part of my teaching life. It had really started to drag me down. And I didn’t want to approach assessing student learning as a soul sucking slog, not just for personal reasons, but also that it was interfering with my ability to accurately assess. If I was absolutely dreading reading a stack of essays, then something was amiss, and I wasn’t going to be able to, in my view, really accurately assess what has this student achieved.

Rebecca: About you, Maggie?

Maggie: I think really a lot of what Jessamyn said echoes in the reasons I decided to incorporate the unessay in my classroom. I was actually inspired by a talk that Jessamyn gave to our faculty, I guess it was over a year ago now, about the unessay and I immediately thought of my senior seminar class. So in criminal justice, in our department particularly, it was common for students to write a 20- or 30-page long research paper, and they dreaded it. It was a source of anxiety for students. It was something in the end that I’m not entirely certain they felt proud of. They just were trying really hard to get the A in the class, to make it perfect, to spend lots of late nights trying to perfect this project and I don’t think it brought about the kind of learning gains that I was hoping to see with my students, it seemed like there was a lot of going through the motions. And the unessay just gave us an opportunity to really turn that on its head and give some of that autonomy back to students and some of that creativity back to learning.

Rebecca: I’m loving these themes of joy, creativity…

John: …which are not terms we often associate with either writing papers as students or with grading them as faculty, and Jessamyn, you were the editor of a fall 2022 issue of Teaching History that focused on unessays. Could you tell us a little bit about ways in which you and other historians have been using them in your classes… some examples of ways in which students have demonstrated their knowledge?

Jessamyn: Sure, and I’ll just give a little promo here, Teaching History: A Journal of Methods is an open access journal. So you can just Google Teaching History: A Journal of Methods, and it’ll take you to the open access platform. It’s hosted by Ball State University. And so that issue is still totally available for anyone interested in reading it. I would say, like some specific projects that were explored in that issue, there was an article about using maps as historical documents, there were a number of projects that were exploring how to present history and historical thinking in a public history setting. So along the lines of museum exhibits, and especially in a digital format, the upsurge in digital humanities is something several of the authors were looking at. And then there’s a whole range of examples that were discussed in the two interviews I did for everything from board games, to graphic novels, to quilt, to the annotated recipes. All those were discussed in that issue.

Rebecca: What are some specific projects, Jessamyn, that you’ve done in your classes,

Jessamyn: A couple that have come up that I used… it’s interesting… Maggie used the unessay with her senior students. And I have helped students in the history department here, they also have to do a major research project. And I’ve been the advisor on a number of non-traditional projects is what I called them at the time, and now would say unessay. So at the senior levels, students did major research projects, like a fictional interview with a historical figure, historical fiction would be another example. I’ve had a couple of students do online exhibits and one person do an in-person exhibit. But I also use the unessay assignment in my intro level class, which is a basic history survey class. And those projects ranged from again, the board game, I had a number of posters, I had one student who was a trained professional baker do a set of cookies with icing, comparing different protest movements, the Bonus Army March in the 1930s with the more recent climate change activism. And one of my very favorites was a video recording that a student did, showing step by step how she made a family recipe. And since the theme of the class was the American Promise, and the way that throughout our history, as a nation, we’ve struggled to achieve the ideals of equality and liberty for all. And we get closer and we fall short. And her recipe was from her family’s background from India. And she took us step by step through it. She had music playing and was describing the importance of this particular dish in her own family history. The unessay was encouraging them to see their location in the American Promise. And it really struck me, in my previous life as a historian of food and food waste. So that was interesting. But most striking to me was, this was a student who had been very, very quiet in class. I felt like I didn’t know her very well. She was doing great work, but she was sort of on the edge of things it seemed like, and with this unessay, it completely changed my perception of her. I felt like I got to know her so much better and so did the other students as well. Having students share their unessays is an important part of the work and It really changed just the whole classroom dynamic in that way. So that was really notable.

Maggie: Nice.

John: Maggie, what are some of the projects that your students have done as unessay assignments?

Maggie: Yeah, so I mentioned before that I assigned the unessay for my senior seminar class. So it’s a sort of capstone project. And there were some really awesome projects that came out of this and I probably don’t have time to talk about all of them. So cut me off at some point when I’ve been going on for too long. But I had a student who wanted to be a elementary school teacher, and had gone through our program and decided that being a practitioner in the field of criminal justice was not exactly where they wanted to be. And so his project was to create lesson plans for second graders to communicate some of the harms of incarceration and how it affects youth when their parents or caregivers are incarcerated. And so it was a really wonderful project, all the students had worksheets, kind of like you can think about having when you were in second grade, and little matching puzzles and games, and it was a really cool experience. And for him getting interested in a master’s of education program, it was a way that he could then take that project and show to his grad school applications and advisors, like, “Hey, this is something that I’m really passionate about. I’ve been working towards this. And yeah, it was a lot of fun.” We had a good time, even some spelling words thrown in there. [LAUGHTER] It was great. Another student project was a choose your own adventure story. There’s various examples of this. But in my youth, it was the Goosebumps choose your own adventure where you read to a certain part in the story, and then it gives you the option of what you want your protagonist to choose. And then you have to flip to a different area of the book and see the fate of that character. And so the students started a really great project, unfortunately, didn’t get to finish it for various reasons, but started a really great project about reentry. And so the main character was someone who was being released from prison and had to navigate all of the barriers and circumstances around reentry that made it difficult for him to assimilate back into society. So that was a big fan favorite amongst the class. They really enjoyed that. Of course, we also had some board games, one in particular, was a monopoly game that examines varying socioeconomic status, and how that influences someone’s path through the criminal justice system. So the players started off with different amounts of money. And as they would pull community cards and have to purchase property and such, they would be challenged to need to pay a particular court fine, and how that would set them back from being able to find a job. It was a very well thought out game of Monopoly. But the goal to show and highlight the inequalities related to income and how that affects criminal justice processing. I could probably keep going, but it really gave a lot of opportunities for students to combine what they were interested in. There was a student who had run their own podcast for a period of time. So she conducted a podcast series on Amber Alerts and their effectiveness. And it was something that when she had been traveling down to Texas, and had received a number of Amber Alerts in the short time that she was there, she was like, “Are these things really working? And how can I explore that through a medium that she was already quite drawn to?”

Rebecca: Unessays always sound like so much fun?

Maggie: Yeah. And they’re fun for us. I know Jessamyn touched on that, too. It’s a whole different experience as someone assessing the work that they’re doing when you know that they’re passionate about it. And it gets me excited about assessment, which I don’t think is something we can typically say.

Rebecca: There’s not a big line of faculty raising flags saying assessment is fun.

Maggie: Right. [LAUGHTER]

Jessamyn: I also wanted to follow up on a point Maggie’s touched on a couple times about student autonomy and agency. I think for both our student populations, this is a really key academic, professional, and personal skill that many of our students need to build, that they arrive to college with a sense of themselves as very passive recipients of education and have not had the opportunity to really see themselves as producers of knowledge. as having agency in academic settings. So for me, one of the great glories of the unessay, is that it’s starting point really is, what do you want to do? What do you want to do to demonstrate that you understand this topic, and for a lot of students, it’s scary at first. A few might think that you’re trying to trick them. And I know this was something we did want to talk about. It does require, on our part, a fair amount of structuring and transparency and communication with students, maybe even especially the highest achieving students. One of my favorite students of all time went on to become a high school social studies teacher. And she did a fantastic job in the intro class with the unessay and then told me like a year later, “You know, I hated that assignment when I did it.” And it was because she was very, very nervous. She was a straight A student who was really nervous about doing something different. But she said, “Now, it’s my favorite, looking back.” So providing that structure and support and transparency, a lot of careful thought on the front end.

John: And one of the things we’ve been talking about in one of our reading groups, Mind over Monsters this semester, at both of our institutions, is the importance of autonomy, in terms of increasing student motivation and giving them more sense of control. It sounds like it’s really helpful in motivating students.

Jessamyn: Yeah, I think so. In both of our disciplines, criminal justice and history, there is a major learning curve to helping students see themselves as having agency in this particular discipline, especially the way they may have interacted with it before. And for me, coming from a lot of their previous experiences with history has been learning what happened and not actually doing history or thinking about those processes and the unessay could really help them move towards that particular desirable learning outcome.

Rebecca: I think one of the challenges that faculty who have been engaged in unessays before see before they enter in is the complexity of having students doing many different kinds of things at one time and managing those projects and also assessing those projects. Can you talk a little bit about how you help provide that structure for yourselves and for students?

Maggie: Yeah, so for my criminal justice class, I spent a lot of time the summer before I implemented this project, creating very detailed instructions and expectations. Jessamyn, I went to your website, and I use some of the self assessment tools, and I adapted them for my class as well. It was a very mindful process to think about how to give students the structure they need to feel confident in their ability to carry out this project, but also give them enough flexibility and creativity to take this project where they want it to be. I also spend a lot of time reading about different alternative grading practices to also incorporate in this, so I use kind of a combination of ungrading and specifications grading as well to help students assess their work. And I met with students quite often throughout the semester, particularly we had a midterm and a final exam meeting where we talked about the strengths of their projects, where they needed to go and what areas they needed to work on, whether it was engaging more deeply in the research and how to communicate that effectively in their projects. It was a lot of work for the faculty, but it was worth it in my opinion. It wasn’t so onerous and it wasn’t boring. It was exciting to meet with students, even if it meant spending my entire midterm week in the office with the students. It was almost every minute of my day for an entire week doing this, but it was, I think, a unique opportunity for making those connections with my students.

Jessamyn: I have been lucky. I have a relatively light teaching load and small classes. So that did make a difference in how I thought about structuring and implementing the unessay. There would be challenges on a larger scale. I would also say that there’s components of the unessay, that it’s not just do anything you want, and then nothing, and then in the end they turn in something. No, there’s stages and components like usually a proposal process and I also was able to meet with students. You could meet with students in small groups as well, if it’s a larger size class to go over the proposals, and many unessay assignments also do have a written component like a bibliography, an intro. At the midterm and end there could be a reflective assignment or sort of summary of work achieved. I did co-create with students the assessment guidelines for their projects. So taking the autonomy and agency a step further, I didn’t leave it totally up to them, we worked together on it, but asked them, “So this is your idea, this is the project you have in mind. How will it show these important skills? How will it show that you’ve met these course learning objectives?” And we work together on it. And that part is always great for me, I just really enjoy trying to de-center myself a little bit as the instructor which again, is something I can do from a place of certain privileges. I have tenure, I’m white, I’m cisgendered. But trying to enlist students as partners more, decentering myself a little bit, and asking them to reflect on what would show that you have achieved this learning outcome with this unessay assignment. A nice bonus is always students, at least one student, will say this is so hard, teaching must be really, really hard. [LAUGHTER] So bringing them in on that assessment process can help them even reflect on what does grading and assessment mean? What would show me that you’ve successfully achieved this learning, this desirable outcome?

John: Do either of you use rubrics for the unessay assignments?

Jessamyn: Well, when I co-created assessment guidelines with students, some of them used a rubric form. A lot of my students found rubrics helpful for things like deadlines. And I did set deadlines for certain benchmarks. And that was something that was easy to map out and help them structure their work time and stay on track as well.

Maggie: Yeah, I created a checklist with that self assessment that was on Jessamyn’s website, geekypedagogy.com. I did have students go through that and rate themselves, and then we got together in a meeting to discuss where we felt the accuracy of each of those points. But I think it gave them some comfort in saying, “Okay, I know I did multiple revisions of this, and I incorporated all of the feedback I was given. And here’s the changes I’ve made, or here’s the areas that I think I probably could have done better, or I could have focused my time a little bit better in this way.” And it just gave them, I think, a nice guideline to be able to talk about the strengths and areas of improvement for their project.

Jessamyn: I’ve used rubrics before, when I was co-creating the assessments. That’s fine with me, if that’s what they want to use. I will say that this part of it, while being very cognizant, like I said, of my own privileges and positionality and aware of that, your teaching context can really shape what kinds of assessments you can implement, for sure. But I also think that it’s at least worth considering the idea of our academic egos and our investment in our discipline and expertise. And what the unessay may challenge us to do is let go of the thing we know how to do absolutely better than anything in the world. So as a historian, a trained historian, I know how to assess student writing better than any other task I do as a teacher. I know so much about it. And it’s easy for me to do, not joyful, [LAUGHTER] but not difficult. It doesn’t push me to think about anything outside my comfort zone. And when I assign an unessay, I have to let go of that and think about things differently. And again, keeping in mind that I know bandwidth is limited, the incredible pressure on a lot of people in the pandemic pivot and for all various reasons, our limited energy, it might be too big an ask, too big a lift. But I would encourage listeners, if it’s possible, to consider maybe it’s not about me. [LAUGHTER] It’s not about us, and what makes us most comfortable as the person with the PhD in the room. That being a little uncertain. how can I assess did students meet this learning outcome with a quilt, with a board game, with a podcast, that it’s okay for us to feel a little bit discomforted. That’s about maybe getting out of the students way a little bit.

Rebecca: When I hear folks talk about unessays, both of you as well as others that I’ve heard talk about unessays, I see a big emphasis on process and the behaviors of a discipline, rather than necessarily the outcomes of something. So often in a research paper, we might assess on some of the polish of something like that, rather than really seeing always the process that goes into the thinking. But I feel like a lot of times with these projects, you end up knowing a lot more about a student’s thinking, because you’re talking with them about their projects in an ongoing way. Can you talk a little bit about that process piece with unessays?

Jessamyn: Well, one thing that jumped into my mind, Rebecca, when you were asking your question is writing a thesis statement for a research paper. Now, this happens to be something I’m incredibly good at helping students do, crafting their thesis statement. But, just like you described, it’s often so much about the academic writing skill that I’m helping them with, and when they have to write a thesis statement for the research paper they’re working on, that’s a great skill to have, if you’re going to be an academic. If you are engaging in scholarly writing, you have to know how to do that. And for a few students, that’s important. And that’s not like written communication skill. I’m talking about the highly academic scholarly skill of summarizing the argument you’re about to make in a paper. So for students who want to go on to be graduate students, I’m all in. We’re going to sit and we’re gonna go through this word by ever loving word to figure out the best way to state your argument. For a lot of my students, though, is that the best investment of their time, and the best way I can help them? There might be other ways to, just like you said, have them articulate and demonstrate their thinking, which, and I’ll just throw it out here in the age of generative AI and ChatGPT, this is at the heart of how we help students navigate is their original ideas really matter, their ideas, and their thinking matters. So if I can support a student being able to demonstrate their thinking and ideas in a way that matters to them, that they feel articulate doing, then, mission accomplished. That’s what I want. That’s what I want for students.

Maggie: Yeah, and I would say that, in my class, the 20- to 30-page research paper does serve students who are going to go on to graduate school. But like said, I don’t know if that is the best way to help them make the connection between what we’re doing in the classroom to what they are going to be doing on a regular basis in their various careers that they find themselves in. I wanted to make this project less about a means to an end for them, I wanted them to really think about how they start with an idea and how if they spend enough time thinking about that idea, and really focusing on how to communicate that expertise that they gained throughout the research on this idea, that they can create something that they’re proud of and they are hopefully less focused on the grade at the end of the semester and more focused on “Okay, I was actually able to start in one place, develop that project, develop this in a way that I can now go to my future employer or in an interview and say that I’ve developed these skills along the way that are mine and mine alone. They’re not ones that have been imposed on me by an instructor.”

John: We have increasingly diverse student bodies in terms of their prior preparation. Some have come through and had lots of practice writing throughout their high school careers and have developed those skills, others haven’t had as much preparation in writing. And we have lots of courses, though, that give them that but writing skills are not always one of the most important learning objectives for many of our classes. And it seems to me that perhaps using unessays can help provide a more equal playing field for students who may not have as strong of writing skills, but might be fairly comfortable creating videos or might be able to engage in some other creative activity that can demonstrate what they learned much more effectively than if they were constrained to having to do it in a written format.

Maggie: Well, I would say that was my experience for sure. I had a group of students make a series of TikTok videos and so they were quite excited about their ability to use something that they are on almost every day, if not every hour of their day, to use that as a platform to communicate the time topics that they were most interested in. And so it did mean that I had to learn a little bit about TikTok, I had to create a profile so I could follow my students and be able to see the videos that they had created. There’s an up voting system, too, I had to navigate [LAUGHTER] but yeah, in terms of equity, it does give students the ability to make decisions about how to present information in ways that they’re comfortable, but still challenging. I think that no matter what mediums students picked, it was challenging regardless. So it didn’t feel like students were just picking something that “Oh, well, I do this all the time.” It really was a strong motivator. And I think in a lot of the conversations we’ve been having in our reading groups about how to channel anxiety about something into a more direct way to accomplish a challenging task. I think it gave even students who are perhaps underprepared for academic writing, or find a lot of anxiety around that to shift their focus away from what they envision that they struggle with to then start in a place of, “Oh, I know I can do this, and I can communicate it in this way.” So I think it gave them some of that confidence back, that perhaps they find disappears when we impose a pretty lengthy paper or academic medium on them.

Jessamyn: There’s two aspects of the unessay and equity minded teaching I’ve been thinking of. One is the universal design for learning concept of providing some choice. It doesn’t have to be unlimited, it doesn’t have to be infinite choice. But providing some choice does increase accessibility. The second kind of key equity minded principle I have in mind with the unessay is that we should not be grading and assessing people’s life circumstances, we are grading and assessing their learning and their ability to show that they have reached certain benchmarks and outcomes. Just to put it in here as a caveat, I do think all kinds of writing is very, very important. Writing skills are very, very important. And a lot of the unessay assignment as I use it does include some clear written communication, because that is a skill we need in this world of ours. So the equity piece comes in though, that if I’m going to be assigning a grade assessing student writing, then it’s on me as the instructor to teach that and create the conditions in the classroom, the practice and the feedback, and more practice and making mistakes, and more feedback to do that writing piece. And if I’m taking into account the wide variety of academic backgrounds, what needs to happen in the course, for students to be able to demonstrate their learning accurately via a written essay, that’s a lot of writing work that I’m going to be doing to provide that equitable classroom. So the unessay as an option, that’s something a lot of people do offer the unessay as an option alongside a traditional essay option, the unessay as an option could increase equity in that way. I think it does convey the idea that our diversity is an academic asset. That all said, it’s interesting that the unessay could also exacerbate inequities without structure, without support. Students who have had very rich and varied past educational experiences may leap at the opportunity and see it just solely in terms of its benefits. Other students with less of that in their background might be extremely nervous about it as a whole new frontier for them. So without some structure, transparency, without all that prep work, Maggie, that you were describing, it could actually exacerbate some of those inequities.

John: And I think the scaffolding that you both described is really important in relieving some of that uncertainty, because I know some students when I’ve given similar types of assignments are really anxious about it and letting them do it a little step at a time and master that step seems to help relieve the pressure on them by just reducing the uncertainty.

Maggie: I know one of the things I incorporated in the project, and I think Jessamyn did this as well, that I gave students unlimited rewrites on any of the written work. So I did have a bibliography and I did have a proposal and a sort of artist’s statement reflection at the end of it. I think, again, I took that from geerkypedagogy[.com] and I gave students the opportunity to just keep rewriting and rewriting. And yes, it was a bit more work on my end. But it was so refreshing that students were actually taking my feedback into their work and they were making improvements. I think that whenever I’ve done drafts in the past, they might go through and make just the easy corrections and leave some of the harder things aside. But in this project, because we started a lot of those elements very early in the semester, we spend weeks and weeks just revising. And it really did give students, I think, a opportunity to make sure they fully were understanding and achieving those outcomes.

Jessamyn: Yeah, that point Maggie about your feedback actually being read and applied is so important for students and for faculty. It’s discouraging and disheartening to provide students a ton of feedback and then feel like it hasn’t been read. It’s up to us, though, to provide the opportunity for students to apply the feedback. If it can’t be applied, it’s very difficult to take it seriously [LAUGHTER] and to see it as meaningful and relevant. And I think another aspect of what you are describing that listeners might be interested in… if we haven’t totally convinced you yet [LAUGHTER] to use the unessay… is that it does get students’ attention, it cuts through the static. We get into a routine we’re just trying to get through, check that box for students who are overworked, overloaded, for whatever reason. I really feel like it kind of cuts through a fog that many of us, students and faculty, are in just by asking us to do something a little bit different.

Rebecca: Sometimes in those cases, and you guys have alluded to this before, that doing something different if they’ve not done it before, is risky and scary. And so all that scaffolding that you’ve built described as so important for students to be able to take that first risk and maybe take other academic risks in the future.

Jessamyn: Yeah, we can’t underestimate the impact of our traditional grading systems and traditional school systems where the stakes are sky high. You can’t overestimate them, financial aid, everything, is so tied to a student’s GPA, their families support, staying in school is often tied to GPA, their own sense of their selves as students, who they are, is defined by the grade they get. So asking them to, Rebecca, like you said, consider the process, not the end point. It is a big ask. So we can’t expect instant student buy-in, that they will immediately jump on board and with ease and joy, even if we are feeling that way about it.

John: So it sounds like it’s been a lot more fun, even though it may sometimes take more work for both of you. How have students responded? You’ve mentioned a little bit about student responses. But how do students generally respond at the end of the semester? Is it something that they find to be equally joyful?

Maggie: Jessamyn touched on this, but there was a bit of student buy-in that I had to get. And there were a couple of students who still wanted to write a traditional research paper. But in the end, they did a podcast element as well with it. So it was the sort of combination of the two, but it was the first time I had ever implemented an unessay in my classes. So I didn’t have a lot of student examples to show them. But I did spend a lot of time creating my own examples of what an unessay would look like. And I think for each of those scaffolded assignments I created a bibliography and a presentation and I tried to create an unessay myself so that they could see what it would look like. I think that gave them a little more confidence in knowing what was expected of them. But really, it took a lot of conversations with students on why I was doing this, what I thought the importance of shifting away from a traditional research paper, what gains they were going to have from that, and every student, at least to my knowledge, really enjoyed the process and much preferred the unessay over their 20- to 30-page research papers.

Jessamyn: The student resistance you just mentioned, Maggie, reminds me that I mentioned teaching context before, but it is important to underscore that any non-traditional teaching and learning practice and assessment, you can expect greater student resistance or reluctance when you quote unquote don’t look like a professor. So when you don’t fulfill the stereotype of what a professor looks like, if you’re navigating any kinds of intersectional biases, you may have to take into account the student resistance component and student biases. It’s a point that Chavella Pittman has emphasized for like an ungraded practice, that it’s very important to take those considerations into account. I will add that the biggest benefit besides my own grading joy is what I see… it’s extremely hard to measure… I couldn’t put it on my end-of-the-year file assessments of my own teaching, but students gain confidence, students gain some agency, some autonomy. I can see it in how they talk about it. If you can have a showcase or a gallery walk, you can see that they are proud of what they have achieved. When we’re doing senior presentations, that’s always very clear, like students’ sense of accomplishment is very notable. And it’s not that that doesn’t happen with the research papers. Well, it can, but the range of students who express and demonstrate to me that they’ve gained some confidence in themselves as students, as thinkers, as knowledge generators, as producers, it’s really heartening.

Rebecca: So we always wrap up by asking what’s next? Jessamyn, you want to go first?

Jessamyn: I’m headed to the POD Network Conference. I’m gonna do a session, with Chavella Pittman and Thomas Tobin, on faculty development that acknowledges teaching inequities. And I am working on a book on teaching and learning when things go wrong in the college classroom, and I just recently signed a contract for that book with the new series at University of Oklahoma press called Teaching, Engaging, and Thriving in Higher Education.

John: Maggie and I will both be at POD and will try to get to your session. And we’re looking forward to that book.

Maggie: I suppose what’s next for me feels like such a loaded question right now because I’m anticipating applying for a sabbatical. And so I’ve been really ruminating on this question of what is next for me. I’m entering into a mid-career stage, and so I feel like my answer is “I’m not quite sure yet.” But as far as the unessay goes, I do plan on bringing the unessay to my courses next semester. I’m teaching two elective courses on death penalty, and women and crime. And I think really any course, but particularly these courses, would be well suited for an unessay project.

Jessamyn: That reminds me, I am lucky enough, I’m going to be going to the American Historical Association Conference this spring and doing a workshop on the unessay, so any historians listening, I hope I see you there.

John: And we will include links to Teaching History, to that specific volume, as well as to the other resources that have been mentioned here. So check the show notes to find more of those resources. And thank you both for joining us again.

Maggie: Thank you so much.

Jessamyn: Thank you.

Rebecca: It’s always wonderful to talk to both of you

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

[MUSIC]

314. Handbook of Online Higher Ed

Since its start in the late 1990s, asynchronous online instruction has spread throughout the world and has been the subject of extensive experimentation and study. In this episode, Safary Wa-Mbaleka, Kelvin Thompson, and Leni Casimiro join us to discuss their new handbook that examines effective practices in online learning from a global perspective.

Safary is an Associate Professor of Leadership in Higher Education at Bethel University in St. Paul, Minnesota. He has authored and co-authored more than 40 scholarly journal articles and more than 20 books and book chapters. Kelvin is the Vice Provost for Online Strategy and Teaching Innovation at the University of Louisville. Kelvin developed the BlendKit Course open courseware as part of the Blended Learning Toolkit, and he co-hosts TOPcast: The Teaching Online Podcast. Leni is a Professor of Education, the Associate Dean of the AIIAS Graduate School and Chair of its Education Department and the Director of AIIAS Online, the virtual campus of the Adventist International Institute of Advanced Studies (AIIAS) in the Philippines. Kelvin, and Leni are frequent invited speakers on topics related to online instruction. They are the co-editors of The Sage Handbook of Online Higher Education.

Show Notes

Transcript

John: Since its start in the late 1990s, asynchronous online instruction has spread throughout the world and has been the subject of extensive experimentation and study. In this episode, we discuss a new handbook that examines effective practices in online learning from a global perspective.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guests today are Safary Wa-Mbaleka, Kelvin Thompson, and Leni Casimiro. Safary is an Associate Professor of Leadership in Higher Education at Bethel University in St. Paul, Minnesota. He has authored and co-authored more than 40 scholarly journal articles and more than 20 books and book chapters. Kelvin is the Vice Provost for Online Strategy and Teaching Innovation at the University of Louisville. Kelvin developed the BlendKit Course open courseware as part of the Blended Learning Toolkit, and he co-hosts TOPcast: The Teaching Online Podcast. Leni is a Professor of Education, the Associate Dean of the AIIAS Graduate School and Chair of its Education Department and the Director of AIIAS Online, the virtual campus of the Adventist International Institute of Advanced Studies (AIIAS) in the Philippines. Kelvin, and Leni are frequent invited speakers on topics related to online instruction. They are the co-editors of The Sage Handbook of Online Higher Education, which we’ll be talking about today. Welcome Safary and Leni and welcome back, Kelvin.

Safary: Thank you.

Leni: Thank you.

Kelvin: Good to be here.

Safary: A pleasure to be here.

Rebecca: Today’s teas are? Safary, are you drinking tea?

Safary: I’m having water this morning.

Rebecca: A key ingredient to tea it might add. [LAUGHTER]

Rebecca: How about you, Leni?

Leni: I used green tea, particularly this Japanese matcha. [LAUGHTER]

Rebecca: Nice. How about you, Kelvin?

Kelvin: I have deconstructed tea. That’s also called water.

Rebecca: [LAUGHTER] Popular globally.

John: And speaking of globally, Rebecca and I are both drinking Moon Bird tea, which is a gift from one of our listeners in France who sent this to us a few weeks ago. So again…

Safary: Wow.

John: …thank you, Myriam.

Rebecca: Yeah, it has a nice hint of pear and elderflower.

John: …which is also a green tea.

Rebecca: Yeah.

John: We’ve invited you here today to discuss the SAGE Handbook of Online Higher Education. Could you tell us a bit about the origin of this book project?

Safary: The origin of this project is actually something that has to do with me having worked with Kelvin several years ago at the University of Central Florida. And right after that, I decided to work in the Philippines and that’s where I met with Leni Casimiro and we worked together. And at both institutions, we were working with online education. And eventually I was transferred to work for two years in Kenya. During the COVID-19. I happened to be in Kenya, and I quickly saw the great need of people wanting to have online education. The resources went up in the place. The things were scattered all over the place. And immediately the idea came that we needed a project that captured the whole world because now this was a worldwide phenomenon, it was no longer something peculiar to Kenya or Philippines or U.S., the whole world was in need of a tool like this. And that’s how I reached out to Kelvin and to Leni.. Thankfully, they both agreed to be part of the project. And I think, from my perspective, that’s where it came from. I don’t know about them… how they think about this? [LAUGHTER]

Leni: Well, for me, it’s really a big project that we did, combining the different parts of the world. You see where Kelvin comes from, representing the West, I represent the opposite, the East. And although Safary comes from the East as well, but he can represent the African continent. And so this really makes the book a global project, really a blend of different perspectives. And so I can say that online learning is represented all over the world in this particular book. And this is indeed, a big surprise to all the readers and a big discovery for everyone.

John: Speaking of readers, what is the intended audience of this book?

Kelvin: Well, I mean, honestly, I would say anyone, anywhere, around the whole planet, who in any way touches online or digital education, should access this book. It’s great for libraries and institutions to acquire and be in their communities. It’s a big book. There’s stuff in there for everybody. So I think it’s a great resource.

Rebecca: Speaking of the size of the book, the handbook contains 50 chapters. Can you talk about how you selected those chapters?

Kelvin: I think the scope and the sequence and the layout of the chapters and the sections sources originally to Safary’s proposal with the publisher, but it was intended to be rather comprehensive with sections like fundamentals and student support and administration and instructional design, instructional delivery, regional specifics, particular regions around the world, and how online education might differ a little bit in, say, the African context versus the European context. But over time, as we were recruiting authors, and as the writing process started, you get a little bit of evolution, the sections might morph a little bit, the distinctives of a given chapter might adjust based on interest and specializations of the authors. So that’s a little bit of the insight into the evolution. But I credit Safary for the vision, which I would say, is probably about 80 plus percent of what he originally had envisioned in the layout. That’s my guess. Safary, would you agree with that?

Safary: Yeah, the thing is that, when you work on a huge book like this, especially a handbook for Sage, they want to have the complete plan when you submit your proposal. Before I can get my co-editors to agree with me, they need to have kind of ideas, okay, this is what I have in mind. So usually, when I work on a handbook like this, I come up with a rough draft. And Kelvin and Leni were very good in catching certain things that I wouldn’t have caught because of their expertise, their experience, and their regions that they represent. And so in the end, what we have here is a product of the Table of Contents was really the product of these three brains that are speaking today.

Leni: I really liked the way Safary has chosen the chapters of this book. Well, we can say that he really originated the choice of these chapters. As you can see, from the perspective of a reader, when you look at the content, you can look in the sequencing, and you will find that you are actually looking into the step-by-step development, or the step-by-step process of engaging in online education. I will say it’s almost like a manual, almost every step that you will go through in undertaking online education in your institution is covered in this book. That’s why it’s really a very important book for every school to have.

John: We had some challenges coming up with a brief intro for each of you, because each of you has done so much with online education in many different roles in many different places. But you also have an editorial board for this book, which is a little bit different than many other books that we’ve seen in terms of handbooks. What was the role of the editorial board in putting this handbook together?

Safary: Yes, we had an editorial board. When you have a project of this magnitude, it is really important to have experts from different parts, especially at the global perspective of experts, and of course, experts on the different topics that are represented in the handbook. As much as we have experience with online education, we cannot assume to know it all… areas where we definitely need help. And so we selected very well known, very well recognized experts from different parts of the world. As far as online education is concerned, all the names that are there are people who are very well respected in the field of online education within their respective countries. The role they played was, for them to be our experts in checking the accuracy and the quality and the completeness of the chapters that were submitted to us. So basically, each chapter went to two to three reviewers and the editorial board members were the primary reviewers to help us really catch everything… and the work they did, I know that some chapters had more feedback than others, but I can say that contribution they gave through their feedback was very substantive in improving this handbook. I don’t know, Leni, how you found that when you’re working with the editorial members who are assigned to you?

Leni: Yeah, actually the editors we chose, I can say they are truly excellent and helpful. During the early parts of the writing of the chapters we lead editors are having like a tug of war with the chapter authors. They tend to bargain their thoughts with us, but when the editorial board came into the picture, it gave a more balanced outlook into writing the chapters. And so we really appreciate their services. The other thing is that this editorial board members are experts in the area and so we can truly depend on them. Their feedback were truly much valued and contributed much to the excellence of the contents of this book.

Rebecca: So the handbook is divided into seven sections. Can you provide a brief overview of each of those sections to give us the lay of the land?

Leni: Oh Yeah, seven sections, it’s nice to give an overview for people to know what the book contains. First section, of course, is the fundamentals of online education. It contains the introduction to the topic of the book, online learning, and some variations in online delivery, like blended, MOOC and ERT, emergency remote teaching, we just really call it ERT, and that became popular during the pandemic. The second section, online education around the world. This section is the most colorful part of the book, at least for me. Because it tours us around the world and gives us a view of how online education grew in varied contexts like US, Canada, Europe, Asia, Latin America, Africa, Australia, and the Middle East. The third section, Online Instructional Design, this section now brings us to the T-cell of online learning, the design of online instruction with focus on how learning happens online. This is now the more serious part of the book. While we came from the most colorful, we now go to the serious part of the book. And then the fourth one, Online Instructional Delivery, this section focuses on the hammer and nail of online learning, the actual online teaching, and this is the most exciting part. Because this is now the delivery, the previous one was the most serious part, this one is the most exciting part. And then perhaps, Kelvin, can you say about the fifth section [LAUGHTER] Instructional Technology for Online Education?

Kelvin: Here’s what I would say about that, if you’ve got the most serious, you talked about the most exciting that you talked about, maybe the fifth instructional technology for online education is the most invisible, maybe that’s what it is. Nobody thinks about plumbing until it doesn’t work. [LAUGHTER]

Leni: Thank you. So that’s technology, I would say this section is essential, because you cannot teach without knowing how to use technology [LAUGHTER]. And the sixth section, Online Education Administration and Management, I would say this is the driver’s seat of online bandwagon [LAUGHTER]. Online education can never prosper without the support of the school administration. So, leading school reforms, like entering the field of online education requires certain strategies to be certain of success. Therefore, I would say this section will indeed equip the readers with those skills, perhaps Safary, can tell us what section seven is?

Safary: I would say the last section is the Customer Service, given that the students are the customers. So the customer service, how to make sure we deliver the best customer service to the online students. And so it discusses all those different aspects of how to really prepare, plan effective service to the students, because many times when people are migrating from face-to-face to online or integrating online education, they forget that online students actually need serious support. And this support definitely needs to be defined. And people who are dealing with the students need to be trained. And so the last chapter actually deals exactly with that.

Leni: For me, because I was looking at the table of contents, and I was smiling in my mind, wow, this is really neatly done. And so this works came to my mind, and I said, Oh, the seventh section, this section focuses on the heart of every online classroom, the students. And so because the students are the reason why we offer online learning, thus we ought to know how we should support them.

Rebecca: One of the things that I love about working on collaborative projects that are really big, and then you have these opportunities to reflect together, is how you summarize what you did. It’s probably really different than while you were right in the middle of it. And it’s fun watching the facial expressions and things as you guys are describing the different sections.

John: With 50 chapters, there’s a great deal of breadth and depth on these topics. In section one, though, you address two topics which are not always considered as part of traditional, at least, online education, which is the use of MOOCs and ERT, emergency remote teaching. But these have played fairly important roles. Could you talk a little bit about the role of MOOCs and Emergency Remote Teaching in the larger environment of online higher education?

Leni: As I see it, MOOC and ERT are connected to the overall theme of the book, because technically they are both delivered online. Online learning can be synchronous or asynchronous. And it’s mostly taken asynchronously while ERT is done synchronously, because it is generally a replication of the face-to-face classroom through the web. However, there are certain arguments in the field as to whether can we classify these two under online learning, because they are believed to not use the principles of effective online teaching. And they say, is their instructional design in ERT? There are more questions to raise to the point that some people believe they should not be called online learning. But for me, we have a common denominator, course delivery through the web. Maybe we can hear from my co-editors here, Kelvin and Safary, what they think about it?

Kelvin: I was thinking, John, when you asked that question, I think the combination of Emergency Remote Teaching and Massive Open Online Courses, it’s part of the popular conception of what online education is, it’s sort of like what a layperson might think, is, it’s just one big thing. So if you didn’t address Emergency Remote Teaching, Massive Open Online Courses, maybe even Blended or Hybrid learning, those mutations, it might not provide quite the same way in for the broadest possible audience. But then, once we’ve ushered you into the house, through the front door, I hope we do a good job of taking you on a more detailed guided tour through the nuances and everything that online education can be, without just being stuck at that surface level.

Safary: If I may add something to the ERT. Personally, the reason why I wanted to see this chapter there was that outside of the United States and maybe Canada, and a little bit of Latin America, when ERT came, Emergency Remote Teaching came, many people call it online education. And as we know, online education, the way we know it traditionally, is much more than translating your face-to-face class to a Zoom class or Google meet class. And let’s face it, that the word there is emergency. This was an emergency modality, which obviously emergency is never the best option, it means better than the chaos that you’re going through. And so many people who didn’t know online education, they came to believe that Emergency Remote Teaching means online education. And many people who were against online education to start with, it was like, “Okay, we have already said that this thing is really bad because it was an emergency.” So it was very important to distinguish what Emergency Remote Teaching is. And in the future, if somebody wants to use that for another calamity that happens, then they know what steps to take, but it does not replace what is known, what we define as quality online education.

Rebecca: One of the parts of your book, The second section is about online education around the world. And getting that tour around the world is not something we typically get the opportunity to have. So can you talk a little bit about what some of the global differences in how online higher education is structured and practiced across continents and regions?

Safary: This section came up as we were trying to make the book global. We really wanted to hear the voices of the people from around the world and not just the United States… the United States being the lead on online education, no question about that. We wanted to know where things are in different regions that were represented. We had to even go online to try to track people down from different countries. It was not easy finding people from certain regions where we didn’t have a network. So as a result, we’re able to bring on board chapters from different parts of the world. We had a chapter from the United States, we had a chapter from Europe, from Canada, from Asia, from Latin America, from Africa, from Australia, and from Middle East. So we were able to see what was happening in each one of them. And these chapters we had, they were kind of similar in a way where we wanted to know what is happening, what are the challenges, what are the achievements that people have in those regions, so that people from those regions who decided to do more work on online education, they have a place where they can learn of what is happening in the whole region from this book. They can have this as a reference to understand what was happening in their region. It is true that when you have one chapter, for example, I co-authored a chapter on Africa, because I was still in Africa at that time. It’s a chapter that’s covering 52 countries, you cannot really cover 52 countries, we just had to have illustration from some of African countries, because there’s no way we have data on all the 52 countries, but at least, there were some common themes that were coming up from a different African countries if I can speak from that specific region.

Leni: I can speak from the perspective of an Asian because I come from Asia. And I would say, we cannot deny that online education started in the West. But because we live in a connected world, it spread easily. Basically, I can see a lot of similarities around the world. The only differences I noticed, because your question says what are some of the global differences in how online education is structured and practiced? Now, I would say the only differences I noticed are the approaches to online learning, depending on the level of their maturity, in using this modality, and the resonance of the context they serve. Institutions that have been engaged in online learning for a long time definitely deal with issues that are different from those of newcomers, the needs of the context they serve also differ, so the strategies utilized also differ. One thing I would highlight, though, is that you can clearly see the creativity and continuity of people in different parts of the world in running online education. And we still can learn from each other. That’s why I said a while ago, the section on the global online education is really colorful.

Rebecca: One of the things that I think is really interesting about that section, is that it can also give us insight as instructors that teach a global audience about what the contexts are that students might be coming from. And that’s something that we often don’t have a little bit of insight into.

Safary: I think that is a very good point. Now that we have online education, people are teaching in many different countries. I remember just a couple of weeks ago, I was approached by one of my former students who wanted me to teach a class in the Caribbean. If things worked out for me, for that class, I would have just glanced at that chapter that covers a little bit of the Caribbean and see what I need to watch out for. So that is definitely a good point for the section on the different regions. In this handbook.

John: When online education first started, there wasn’t really that much known about what would work effectively. And as online education evolved, we saw the role of instructional design become an important part of the practice of online education. And section three deals with online instructional design. And that’s helped facilitate and inform online education, along with a lot of research that’s been done since the early stages. How have instructional design practices evolved since the early stages of online education in the latter part of last century?

Kelvin: That’s a good question. And I guess I’ve been in this field watching this first hand and touching it for about 25 years now. So I sometimes say not exactly the first floor of the building, but just one step above. And what I would say is that when I started in the late 90s, what we saw a lot was adaptation of traditional instructional systems design models and practices, that is constructs that were used quite often in corporate education. See if this takes you back to the past: CD ROM development, military learners. Those kinds of methods, practices, and models were adapted to this online context. And some of that’s constrained, like you’re making a system, like it’s a bounded system that was, quite often the context, like a CD ROM. And now you’re talking about the internet, a network open system. And I remember some of those early days, like, “Okay, what can we learn from these models? How can we adapt those?” Over time though, we learned that this is a unique context, which then began to have its own models and practices and processes and research and iterations and development. And I think of even things like much newer developments, like alongside of constructs like inclusive pedagogy, we see practices and thrusts, like inclusive design, as being a very specialized subset. So we’ve got a very robust research and professional practice literature that has grown up and these, arguably, two and a half decades of online education experience to draw upon. And I guess I’ll just say this, about that. Throughout my time in this field, what I’ve seen is that online tends to make the formerly invisible, visible; formerly implicit, explicit. And I think that evolution of instructional design and development field, it has learned from that. Online education has drawn us along in what does it mean to bring learners in from really anywhere and bring them together in a learning community, and how do we excel in that. That’s been a really rich progression over these last two and a half decades.

Safary: If I may add to that, the reason why we had this section was that many people who are new to online education, they think that online education is about uploading all the files that you have been using face to face, and then let the students read that, and that’s online education. It leads to a lot of frustration from the students because there was no instructional design for online learning. And so we needed to have a section that would guide people into that. And also for instructional designers in college and universities where they already have instructional designers. Some of them have not gotten a degree in instructional design. So they have limited knowledge. They just happen to know a little bit more than everybody else, but they don’t really have a solid foundation. And so that section helps to kind of guide people in the proper instructional design for online learning.

Rebecca: So sections four and five focus on online instructional delivery and instructional technology. These are topics that we love to talk about and have episodes of this podcast on. But given the time constraints, we probably can’t dig in fully here. But can you help us identify some of the most important changes that have occurred in how well designed online courses are taught?

Leni: That’s a nice question. Kelvin also said a while ago, he was mentioning about the early years of online instructional design, I would say, perhaps 1998 to 2000, those are the early years I’ve been involved, still in the planning stages of online delivery. Most of the online courses we developed were primarily text based, and are delivered asynchronously. That was after the military, Kelvin used, online learning, it was already in the university. Why text based? Because even our students, in the context we are serving also did not have the capability or the capacity to access videos or higher level technology tools. That’s why we designed the way they can access us. And so, yes, it was primarily text based and asynchronous. However, through the years, I would say two forces caused the major changes in the way we design online courses, first, technological developments, particularly in instructional technology. And second, changes in the needs or nature of our stakeholders, the students. Well, technological developments without a doubt have increased the repertoire of instructional media that we can use in designing truly engaging online courses. But as I’ve said earlier, technology is not the heart of online learning… it’s our students. And we saw how the nature of our online students change over time as well. While many of them were happy with plain text based asynchronous online courses during the early days, now they want more real-time meetings. And the flexibility they want is indeed tremendous, I tell you. We notice that there is a greater demand now for more flexible and personalized learning approaches. And these topics are dealt with in this handbook. I know Kelvin has written on this. And some other chapters also addressed this flexible learning, personalized learning approaches. These are now the needs and demands of the new generation of online students.

John: This is bringing me back to a time when I started back in 1997 teaching online when many of the students had 300 baud… [LAUGHTER] …or 1200 baud modems, and you couldn’t do much more than text. And I remember putting in some flash-based videos, and many students couldn’t access those because they didn’t have the download speed, especially students in more rural areas. So there was a lot of resistance to online education when it was first introduced, which is one of the reasons why I think instructional design practices became a part of early online education to help ensure the quality of that. And we do have, in most institutions a fairly elaborate process of instructional design assistance and instructional design review for online courses, which is something that’s never really happened in the same way for most face-to-face courses. Might it be time to start applying some of the techniques and practices of design that’s being used for online course delivery to in-person course delivery?

Safary: I remember about 15 years ago, I was training faculty on online teaching in the Caribbean. And I remember many of them, at the end of the training, saying, “I have improved my face-to-face teaching because of the training that I have been going through for online teaching.” So I definitely believe that if people get the proper training in online teaching, they can use that knowledge to improve face-to-face teaching. Because let’s face it, many people are teaching not because they have a degree in education, but because they have a degree in whatever field they come from, they have never learned how to teach. And so when they go through the training for online teaching, they discover a lot of principles that they should have even been using face to face. So I definitely agree with you on that one.

Kelvin: Yeah, it’s true. I say it all the time online makes the formerly implicit, explicit; formerly invisible, visible. And I think that’s why online has been a vehicle for applying thoughtful design and teaching practices and the improvement thereof. Once you sort of concretize the elements that make up an online education experience, then you can see well, how are they arrayed? Are they lined up properly? Does this cause lead to the desired effect, and you can work on improvement, no offense to anyone in this, but when we just are dealing in the ephemeral, we will walk into a space, four walls and a door, and we say words into the air, it’s much harder to see how those parts fit together or don’t. And it’s harder to be reflective. So, I think that’s the reason that online education has brought more emphasis to potential improvements, continuous improvements, and so I welcome it as a vehicle for a more thoughtful process in general. I love this elegant turn of phrase Caroline Boswell says she frames teaching as a student success intervention. Or as I put it, I’m one of those odd people who sees a connection between teaching and learning. And not everybody does.

Rebecca: You’re kind of queuing up our next question perfectly Kelvin. The final section of your book is really about student support. And our students are often distributed when we’re teaching online. So what are some of the biggest challenges in terms of supporting students that are in these online programs or online courses?

Kelvin: Yeah, I would welcome Leni’s and Safary’s viewpoint on this as well. But to me, I’ll keep it simple and say that the biggest challenge is the diversity of student profiles. The different backgrounds, the multifaceted demographics, and resource or not resource, or technological connection or not technological connection, that diversity makes it awfully hard to assure kind of an equitable experience for everyone. So that’s the gap that emerges, that student support is trying to offer… not to mention the diversity of approaches to design and development in the actual experience. But I’m curious what Leni and Safary would say to that.

Leni: I would go for the opposite, on the side of the teachers, I would say the greatest challenge in student support is personalizing your support. It’s related to your diversity. Almost every online student has her unique needs and contexts. So considering different personalities and backgrounds as well, you may be able to personalize your support. But in the name of efficiency, you’ll find yourself dehumanizing the process. What do I mean by this? Well, machines can never replace human touch. And human touch is what every online student needs.

Safary: If I may speak a little bit from experience I had in Kenya during the COVID-19, we migrated our classes to the online delivery. And I quickly realized that… and this was something that was going on in all of Africa, I know this because I was involved in different international association for online education all over the continent…. and so we were meeting and discussing some of these issues. The major challenges that were going on at that time, I don’t know about today, were dealing with infrastructure, because most universities that didn’t have online education platforms, or online education structural systems, so the technology was not in place. Many students there were not access because the internet was extremely slow, some were using loads of data to access the materials and they would run out. Some had issues with electricity. These are things we take for granted in the West. These are the not issues that we will discuss even in textbooks of online education, but they are real issues that cannot be ignored. And so that was a major challenge in supporting online students, because the infrastructure was not in place. And I think the issue is still the same. But more and more work is being done. I remember, for example, in Kenya, what the government did, they gave the free data access to all the faculty in the whole country, as long as it was used only for instructional purposes [LAUGHTER]. If you want to use it for something else, it wouldn’t work. I mean, that was quite creative, to try to help people to help education move forward, because everything was just stuck because of COVID-19.

John: Over the past year, we’ve seen a fairly explosive growth and use of generative AI large language models, including chat GPT, Claude, and a few others that have come out very recently. And that opens up a lot of interesting opportunities, but also some challenges for online education, particularly concerning the assessment of asynchronous learning. How do you see online education adapting in response to the widespread availability of tools like this, which will only become more powerful over the next few years?

Kelvin: It’s sort of the very definition and epitome of disruptive innovation or disruptive technology. And just to be clear about this, I don’t think it’s limited or focused on asynchronous online education, I think it’s everything. For me, it’s really an opportunity to address learning and assessment of learning much more meaningfully, and I’ll use one of Leni’s words, more personalized and relational. I think one of the things we’re seeing with the injection of these various forms of artificial intelligence into the learning setting is the value proposition of the human. And I think it was Cathy Davidson, years ago, from HASTAC said something like, “If we faculty can be replaced by a computer we should be.” That is, if all you’re offering is something that is easily rendered more efficient and scalable by a machine then, well, what are you doing it for? I think that the opportunity to really gauge learning, which is a very personal and a meaningful thing, we act like it’s something that’s kind of homogenized and industrialized, but learning… I don’t know what learning is, frankly, I can’t crack open a human and see what all is happening with the connections and making of meaning in all the background experiences. All I can do is get insight, but in dialogue, in the creation of artifacts I get a glimpse. If we’re product oriented, to the exclusion of the process, and to the exclusion of the human context, well, that can be certainly disrupted, maybe stolen by artificial intelligence in machines. But if we keep the emphasis on humans, on “Well, John, tell me about this…” that’s more meaningful. I learned a practice a long time ago from a faculty member that I studied under, where she adopted a practice of a learning summary. And in any course, again, that’s just one artifact, but it gives a glimpse into the articulation of what learning is really about. So I think we need to push the envelope in “What does authentic assessment mean? What does meaningful learning look like?” Now, that’s hard to do at scale. Are you going to have a personal oral defense with every student for every assignment, probably not. But if we see artifacts, and products, as breadcrumb trails leading to a destination of a more substantive dialogical process, well, then maybe that’s something. So I don’t think we know yet how this is going to play out. And I think your listeners are gonna find cold comfort from me in getting to an easy solution. But I think the future of responding to generative AI is to lean more into the human and the relational than less.

Rebecca: So we always end by asking what’s next?

Safary: Well, as far as this project is concerned, what is next really, we want to continue building a community of online higher education scholars, practitioners, so that this momentum that has been created by this book can continue, because this is one of the few maybe rare books that really have so much global contribution to online education. Many of the books that are written, they’re usually kind of regional to a specific region of the world. And so this is the first time we have a network of, I think, around 100 people who contributed to this, coming from many different countries. And I feel this has created synergy on the discussion of online education in a way that we should not let that go. So one of the things that we have been talking about is the possibility of holding a summit on online higher education in the next few months, once everybody has gotten a chance to hold a copy of this book, and to bring different experts together from different parts of the world, and try to address online education from different parts of the world, while addressing common issues such as assessment, which is one of the major controversial issues anywhere have been, everybody talks about the challenges of online assessments. So that’s things like this, and probably this artificial intelligence, which is a new thing, we may want to go deeper into that… we’re not able to dig too deep with that, although we addressed it in the book. But we didn’t go too deeply because it was still kind of new ChatGPT was just coming out when we were finishing the handbook. And so that is one of the things that we are looking into, there is another handbook in the making with SAGE that will focus specifically on instructional design in higher education. So that would be like an extension of this project. So we want to continue building on this work, because we consider it’s very important.

Leni: I’m really optimistic about the next steps on this because it’s like a seminal book that really got there’s a global perspective, as Safary says it’s not the same as the other online learning books. So we can also see a lot of developments coming up. And so I will say, this book is just step one, the next steps will really be coming up definitely, because the field is always growing. We have seen its growth, and it will still grow. And so there’s more to follow, I believe.

Rebecca: Well, thank you all for joining us. I know that our listeners will really enjoy the handbook and all that it has to offer.

John: Well, thank you, and it’s great talking to all of you and we’re looking forward to reading the book.

Safary: Thank you so much for the opportunity. Really appreciate that and wish everybody a wonderful reading experience.

Kelvin: Thanks for having us, Rebecca and John.

Leni: Thank you very much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

312. Alice: Finding Wonderland

Many of our disciplines are unfamiliar to students until their first encounter in an introductory course. In this episode, Rameen Mohammadi joins us to discuss his first-year course that introduces students to computer science using an approachable hands-on experience.

Show Notes

Transcript

John: Many of our disciplines are unfamiliar to students until their first encounter in an introductory course. In this episode, we look at a first-year course that introduces students to computer science using an approachable hands-on experience.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Rameen Mohammadi. Rameen is the Associate Provost at SUNY Oswego and an Associate Professor of Computer Science. Welcome, Rameen.

Rameen: Thank you. Thank you both.

John: It’s good to have you here. Today, for the first time ever, we’re all drinking the same tea, which is Chimps tea, a black tea with honey flavor, fig, and thyme. And this is a gift from one of our listeners. Miriam in France. Thank you, Myriam.

Rebecca: Yeah, so far. It’s really tasty.

Rameen: Yeah, really excellent tea. We love it.

Rebecca: So we invited you here today, Rameen, to discuss your First-Year Seminar course that combines animation and storytelling using the Alice 3 programming environment. Before we discuss that, though, could you provide an overview of the goal of first-year seminar courses at Oswego?

Rameen: This is not a standard first-year seminar. First-year seminar courses are designed to extend orientation, familiarize students with resources, and things like that. Our perspective about this type of course, which we call signature courses at SUNY Oswego, is that you are welcoming students to the intellectual community that we have. So as first-year students we desire a number of outcomes to be met by these courses, one of them is critical thinking, they have to have a significant critical thinking component. Also, these courses need to have both writing and oral communication embedded in them. And one of my favorites is that they have to enhance the cultural competency of our students, we’re a very diverse student body, there’s quite a bit of opportunity to make sure students experience other perspectives. And I think courses of this type really need to address that. Our provost, Scott Furlong, brought the idea to us even during his interview at SUNY Oswego, about what they had called, at his previous institution, passion courses. Now, as I said, we call them signature courses here. But those of us who love our discipline certainly can understand when somebody uses the term passion. So what makes you excited about your discipline? That’s what the course should help students experience.

John: So, you’re using the Alice 3 programming environment. Could you talk a little bit about the types of things that students are going to be doing in the class?

Rameen: So Alice 3 is a VR programming environment. So what you do is you build a scene, you can bring avatars of various types, could be people, could be aliens, could be dogs, into a scene, and you have props, trees, mountains, buildings, that you could bring into a scene, and then you learn to program something. So they can talk to each other, they can move from point A to point B. And it actually turns out, they’re able to, and they will be, writing reactive programming, which typically is what we do when we design games. So the user acts in some way, and then you program the reaction in the VR world in that context, or things run into each other. And obviously, when you’re designing games cars, or other things may run into each other, and you have the ability to detect that and actually act on that. But at this point, they are already running about a month ahead of where I thought they could be in just about a month of the semester. So I’m really hoping we can get that far.

Rebecca: Can you talk a little bit about why you chose the Alice platform, and what you were really hoping to foster with students.

Rameen: So, just a bit of background about Alice, Alice is supported by researchers in Carnegie Mellon. I think Randy Paudsch, when he was at University of Virginia, is really the person that began the innovation with Alice. And he thern moved to Carnegie Mellon. Many people in computer science would know who he is because of his work in VR, but what he should universally be known for is The Last Lecture, which is a pretty amazing hour plus lecture he gave before he died from cancer. But that group has been working on Alice for a very, very long time, and of course, has had new actors along the way. Don Slater is one of the people that has been part of that group for a long time, and he’s very much involved and was at the time when I met him very much involved in advanced placement. And that’s something I’ve been involved in for a long time, advanced placement for computer science. So one of the things we do in AP readings, we have people do professional development activities, and he gave a talk about Alice and this is a long time ago. But when I first listened to him talk about it, and he showed the features of the system, I really didn’t have a place for it in anything I taught at the time. So it has been brewing in the back of my head as a thing to build a course around for a long time, but really couldn’t have done it until the opportunity came along to build a signature course.

John: For those students who do go on to computer science. It seems like a really nice way of introducing the concept of object oriented programming. Does it work in that way?

Rameen: So the thing to understand about object orientation is that most of us who are software engineers by discipline, database types, we are very comfortable thinking of a student information being an object and the fact that we have a container of student objects and so on. But it turns out that’s not necessarily as comfortable for students as it is for those of us who do this for a living. But when you say, here’s an avatar, and you put this on the screen, and you could tell it to go from point A to point B, that seems like a very natural idea to students, and the fact that this particular avatar, so suppose it’s a bird, has wings, and opposed to a person who has legs, you don’t have to explain that. It’s a concept as inherent in being a human being and 18,19 years old. So some aspects of object orientation that often is difficult for students that are really obvious in this context. So any object like an avatar of a person, dog, cat, whatever, they can all be moved from point A to point B. So they share a set of expectations and attributes. They have a location in a 3D world, and you can move them from A to B, piece of cake, they understand. But then you say, “Well, this one is a bird, it has wings.” So the fact that you can spread the wings or fold the wings would be a characteristic that exists only because of it’s a bird. So inheritance, which is a concept that we like to teach in computer science is just built into the way the system behaves. And no student will say, “Well, what do you mean, a bird can spread this way or fold its wing.” People just naturally know what it all means. And believe me, it’s not always natural, in some of the other things we try to do with students to teach these topics. So it does lend itself extremely well, in understanding that objects have attributes, they have functionality, and it’s all there on the screen, and they can see it.

Rebecca: I think Alice is really nice, because it is so visual. And so you get those immediate, “I can see the thing I did,” whereas I remember when I started learning some code, I was building a database for car parts, and it was completely abstract. And I cared nothing about car parts.

Rameen: Yeah.

Rebecca: So it didn’t make it that accessible to me.

Rameen: It’s not, exactly, then the other aspect of it that I think we need to think about, about the platform, is that you don’t write a single line of code, you generate 1000s and 1000s of lines of code, but you don’t write any. So if you have a particular avatar as the object that you’re processing at the time, in building your code on the screen, you could just drag and drop functionality it has into your code. If you need to loop and repeat steps, you drop a loop into your code and then put the steps you want to repeat inside that loop. So all the typical barriers a student had with syntax or various languages, whether it was Java, Python, C++ kind of wash away, because you don’t really have to know syntax at all you, need to know “what are you trying to do?” and what will enable you to do it, and then you can execute that. So far, clearly, that’s not a problem for them. Here is the screen, this part of it is dedicated to X, that part of it is dedicated to Y and they’ve been able to handle it probably from week one. So all the standard things that tend to take a long time, don’t take any time. And besides doing 3D graphics, if you are a computer science person, in my mind, is super senior level type of activity. You got to teach them an awful lot about data structures and other event handling elements that they must learn because that is what we all had to learn. But guess what, you can learn it with Alice in short order. And this is the course is proving that you can.

John: Now one of the challenges that I could imagine you might have as that students come in with different levels of prior knowledge or interest or engagement with computer science. Some students may have not written a single line of code in any language, while others may also be taking other CS courses at the same time, or have some prior programming experience. How do you address the differences in background?

Rameen: So my sample case here is small, I only have 17 students, but this is not a computer science required course. So this is one that has art students in it, it has biology students, and and it does have a few computer science students and then maybe this one with an AP computer science background from high school, and none of them are doing any better than these other kids. So I guess the point is, it levels the playing field in a pretty significant way. If you can think a thought you can probably write code in Alice. And I’m finding it quite interesting, since I’m not preparing them for another course… not only this course doesn’t have a prerequisite, it’s not a prerequisite for anything else. So the way I designed the course going into it, I went into it with thinking, okay, so if storytelling and writing a really cool story within groups is the best I can get out of them, great. If I can get them to a point where they can write new functionality for objects, and I can help them write reactive programming so they react to the mouse click or collisions of objects and so on, maybe I’m dreaming, but that would be fantastic. At this point, I’m pretty certain I can get him there without other stuff. But that was kind of the key coming into the course, I walked in with a mindset of being flexible, that if they are struggling, I’m not going to keep pushing it like I would typically do in a CS course, which is partly why you would also lose students, but at least in my experience with these kids, and I can’t say until I teach it again (and hopefully I can) whether it will be the way it works is that you show them how to do something, and then they go to work, and they start doing it and then they make mistakes, and we all do, and then you give them a little bit of a hint about potentially maybe a different technique they could have used to accomplish the same task. I’m just going to give you an example. So you want the bird to go from point A to point B, so it’s on the ground, needs to go up on top of the tree. So Alice lets you put a marker where you want it to go on the tree, because you can’t go to the tree, you’re going on a branch of a tree. So you need to know how to put a marker there. So you put a marker there. And then it just goes from point A to point B, it goes from the ground to the top of the tree, then you say “Wait a minute, that’s really not the way birds fly.” So now you got to figure out, well, how am I going to flap his wings to go from point A to point B, to go from the ground to the branch on that tree? So it turns out and I’ve come up with a solution to this myself, obviously, you can’t really teach these things if you haven’t thought about how you could possibly solve them. And one of my students, after like three weeks of instruction, she figured out how to do what we call in Alice a “do together.” So as it’s moving from point A to point B, the step that is happening at the same time is the flapping event of spreading and folding the bird’s wing and she made it very clear that the bird was flying [LAUGHTER] from the ground to the tree with no interruption. Then we need to talk about well, do they really need their legs hanging out as they’re flying? I don’t know much about birds, but I think they fold their legs back. So now we have to learn how to address some kind of a functionality that is about a part of the body of the bird. So this is the way the learning is happening in the course, kind of naturally, you’re trying to make a realistic action on the screen in the animation. Well, how are we going to do that? Well, we have to now address the joints like the hip joint or the knee joint or the ankle joint to make that much more natural in the way it works. And there’s no persuasion here, the student is trying to make an interesting thing. And then I’m there to help them figure out how do you make that much more realistic.

Rebecca: What I really love about these courses, and in what you’re describing with Alice, as someone who’s also taught code to students, particularly ones that are not in computer science, is that they’re thinking like a computer scientist, and you’re really getting them completely within the discipline, you’re hooking them right in because they’re leading with their curiosity. They’re not satisfied with the way something looks, so they’re digging in and digging in and digging in. And unlike our traditional way of structuring curriculum, where we think this is the foundational information, and this is the next thing we build on, it almost turns it totally on its head [LAUGHTER], and does it like backwards from what we traditionally do. And it’s really fun.

Rameen: Well, I think the students are at the center of that type of a decision, that for years, you see human beings that probably could do this kind of work, but shortly after they try and they get errors after errors after errors, they say, “Hey, listen, this is great that there are geeks like you would like to do this kind of thing. It’s not for me,” and they walk away from the discipline, even though they could have had great contributions in computer science. So for me if some of these concepts are introduced this way, where syntax and semantics, which is typically what slows people down when they first begin, even the systems we use… like how do you type your program? And how do you run your program? …there’s a whole bunch of instruction around how do you do anything. You just go to alice.org, you download Alice 3, but once you do, it’s here you go, you click here, and then you set up your scene; you click here, you begin writing code. Well, how you write code? Well, the object is on the left side, you drag the command from the left to the right, how far do you want it to go? Well, you gotta choose a certain distance that it needs to travel… really, really easy for students to take to right away. And I just had no idea what I should expect. You watch a lot of YouTube videos. I mean, I certainly do when I was preparing this course, of all these different people, young and old, building things and being proud of what they had built. And I thought, if I could bring that to a course for our first-year students, that would be really, really awesome. And I think that’s what has happened.

John: You mentioned that the students are able to interact. Are they all in one virtual shared space for the class? Or do they have to invite the other students into the spaces that they’ve developed?

Rameen: So this is a really good question, John. So when I imagined how the course was going to work, I had to think of a number of things. One, I asked our technology people to install Alice on all lab computers because I can’t assume or assert that every single student that will take a course like this will necessarily have the equipment that could enable them to run it. Even the Mac kids who had trouble at first installing the thing, and I needed people to help them to get it installed, even they could continue to work because we had the software on our machines. The type of collaboration that I advocate for in class is a little untraditional. At least I think you could argue that it may be. So like the other day, I gave them a 10-question quiz. So they answer the 10-question quiz. And then I said, “Find a couple of other people and persuade them why your answers are correct and their answer is wrong.” So now the whole room starts talking about the quiz. I don’t know if they’ve ever had an experience where somebody says, “This is not cheating, what I’m asking you to do.” Who gives a quiz that says, talk to everybody else to see what they answered for the quiz.

REBECCA; John does. [LAUGHTER]

Rameen: And that’s not surprising, but in my mind, is it about the learning process? Or is it about assessing or giving a grade? This is a very low-stake experience. So why exactly would I care if you talk to someone else about it? So why not persuade someone else that your answer is right? That’s a very different tactic than to say, “Do you know the answer? And are you right or wrong?” Persuading someone requires talking to them, requires thinking for yourself, first of all. Well, why is this answer right? And then opposite of that, you hear the explanation. Are you persuaded that what they said is accurate? Or do you think they’re wrong? In which case now you’re giving them back a different perspective, and then they change their answer. And of course, you could change your answer for the wrong reason. That’s just one example. I really want them to collaborate and work with each other. And every time somebody does something interesting, like the young woman who built the code that I had not been able to myself, having the birds fly from point A to point B, looking very natural. I had her come to the front of the room, plug in with a connector that is in the room and show everybody how she wrote her code. And we’ve done that at least a dozen times so far, where people just come up, plug their computer and show everybody their code. So we often are worried about students cheating and using other people’s work. And if it is about collaborative learning, then you really have to cultivate the idea that, you know, that was a really good idea, maybe I can do that. And I think hopefully, the course will continue to behave that way where I’m confident everybody’s learning from it. That’s the concern that I’m the only one who knows something, whoever I am as a student, and everybody’s just copying me or whatever. That is not my experience so far in this course. They’re just trying to do it better, and if you have a better idea, maybe I can take that and move with it. Then maybe I’ll make it a better idea. And you see that also with students. One of them figures out how to make somebody walk more naturally, and then the other one even enhances that, even makes it even more realistic in the way you would walk. And that’s kind of what I like to see happen and is actually happening.

Rebecca: So how are students responding? Are you cultivating a whole new crop of computer scientists?

Rameen: So, this is an interesting question. I am wondering, those who are not computer science students, whether or not they decide that this may be something for them. But I’m also… with Rebecca here, is good to bring this up… they might become interested in interaction design as a discipline to pursue and become passionate about. Those of us who do this kind of work for a living and have done it for 40 years or whatever, the engagement aspect is the critical aspect. If they are really invested in the learning process, they can overcome an awful lot of barriers, that frankly, I cannot persuade you to put the time in, if you’re persuaded that you could do something a little bit better, then I’m done. As a teacher, I’ve set up the environment in the way it should be where you are driving the learning process yourself as a student, and they look like that’s what you’re doing at this point. And we’re only within a month into the course and they are behaving that way. Now whether or not they will continue to take more computer science courses, and get a degree in computer science. I’m not really sure, but, hopefully, if they do interaction design, there’ll be better interaction design students, because some of the structures that they have to learn here would definitely benefit them in that curriculum too.

Rebecca: Yeah. When can I come recruit?

Rameen: Anytime. I’ve already had Office of Learning Services. That was one of the things I wanted to point out, that I’ve had Office of Learning Services, they came and they talked about the learning process. And besides what they have done for me and talking about the learning process, and it’s all research-based discussions, which is really critical for students to hear things that actually do work, and we can prove they can work. I talk about the learning process on a regular basis with them. And I’m very interested in them understanding why we’re doing anything that we’re doing. I mean, they may be used to somebody standing in front of them talking for an hour and I just don’t do that. I may talk for 10 minutes and then have them work on stuff and then as we see gaps in what they understand then I talk for 10 more minutes, maybe. I try not to talk a whole lot. I want them to be working. So the time I’ve spent is on building what they are supposed to do to learn, not so much talking to them on a regular basis during the 55 minutes that the class goes on. Unfortunately, I’m not quite sure that that’s the experience they’ll have in other courses that they take, because to me, there is a freedom embedded in the way the course is designed that is hard to replicate if you have to cover from A to Z of some topic. You get to “H” and people are having trouble, well, you just keep going. Well, I don’t if everybody has the luxury to say, well, maybe we need to pause longer because we can’t get past this point. I mean, what’s the point? We get past this point, you’ll never catch up to where the end is. So I am hoping some of them will decide to be CS majors as a result, but I’m more interested in seeing how they will do if they take more CS courses. I mean, if they take a CS 1 course, are they going to do better than a typical person taking a CS 1 course if they move on and take data structures and other courses we require? Will it come to them easier? I think it’s a really interesting question. And I think there’s a lot of research that advocates for the fact that they will do better, but I like to see it firsthand.

John: One of the things I believe was mentioned in the title of the course was storytelling, what types of storytelling takes place in the class? Is it the design of the scenes or is there some other type of communication going on?

Rameen: So the way you do animation, in general… and I probably should back up and say I spent about an ungodly amount of hours trying to learn how to do this, but I went about it backwards because I went after event-driven programming and interesting things that I knew Alice could do to write games. But then you step back from it and say, well, students can’t start there. I mean, that’s just not a good place. So then you look at the alice.org website, which gives you tremendous amount of resources and say, “Oh, designing scenes happens to be the first thing they teach you to do. Oh, maybe I should learn how to design a scene.” So you put the pieces that you want in your story on the screen, and if you don’t want them to appear, you could make them invisible. But the way Alice works, you have to put all the components in on the front end. It’s a little different than the way we do object-oriented programming. When we do object-oriented programming, you create things when you need them, you don’t think about setting up the scene on the front end. So that took a little getting used to, but that’s what you do. And then the characters you put on the screen can move, they can talk, they can fly, they can do whatever you need them to do. And my biggest interest was storytelling, when I was conceiving of the course was that I really want students, especially those from other cultures, other backgrounds, to tell their story, find a way to tell their story. And this is probably going to be starting as a group project for my students in a few weeks. And we’ll see how that actually goes. And obviously it has to have a beginning and middle and an end for it to be an actual story. But I’m just excited to see what they will actually decide to do and how they actually do it. Along the way, though, they’re going to need some tools. And that’s kind of what my contribution will be in making sure that they can tell the story the way they want.

John: It sounds as if developing this course required you to learn quite a few new things that were outside your normal teaching experiences. Do you have any advice for faculty who are working on the development of similar courses?

Rameen: So for those out there who teach for a living, the opportunity to build something from the ground up, especially something that frankly, when I first thought of it, I thought it’d be a lot easier than it turned out, because there was so much that I didn’t know how to do, but when you don’t know something you don’t necessarily know that you don’t know something. People who were doing it and I was watching them do it made it look very easy to me. But once I began to do it, I discovered how much work there was to come to a point that actually orchestrates a course, I mean something that is meaningful, and has a clear direction to it. So if you have that opportunity, even after 40 years of teaching, to start over in some ways and build something that you feel not particularly comfortable about, I really highly recommend people do that. Because that is what your students are experiencing every single day when they are trying to learn this stuff that you know so well. So having a little bit of a taste of what it takes to learn something you know very little about, I think is critical. So my message to the faculty who are listening, is that if that opportunity arises, by all means, take it.,

Rebecca: You certainly get a lot more empathy for what it feels like to not be an expert, when you’re learning something brand new again.

Rameen: Well, that’s the thing about most of the things we do. I’ve been programming for 50 years. So it’s one of those things where you’re completely in tune with the idea of: understand the problem, solve the problem, whatever. But where should the camera be in a 3D world in order for it to point at the person talking just the right way? I had to figure that out. It didn’t come naturally to me. The first bunch of programs I wrote the camera was always in the same spot and then I began learning that “Oh I have control over where this camera goes. [LAUGHTER] So maybe it needs to be somewhere else when this person is talking versus this other one.” That’s been a lot of fun to get a sensibility back into the system here that this stuff is not as obvious as it may seem.

John: We always end with the question: “What’s next?”

Rameen: So I certainly would like to teach the course more, and I also want to do some presentations with the faculty in computer science, and if there is interest in graphic design faculty to do some for them, too, because I think the platform is extremely powerful. It doesn’t cost anything, the resources that exist have been getting developed for a very long time and they’re pretty mature. And again, back to no cost. We all know how much books cost. You really don’t need one, you just use the exercises that they give you at the Carnegie Mellon site for Alice and go with it. So I really want to advocate for faculty to consider using it beyond just this first-year signature course.

John: Well, thank you. This sounds like a really interesting project that can really engage students.

Rebecca: Yeah, it sounds like a lot of fun. I can’t wait to come visit.

Rameen: Yeah. Thank you both. Really, this was fun. Thanks for the tea also.

John: Well, we have Myriam to thank for that.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

311. Upskilling in AI

With so many demands on faculty time, it can be difficult to prioritize professional development in the area of AI. In this episode, Marc Watkins joins is to discuss a program that incentivizes faculty development in the AI space. Marc is an Academic Innovation Fellow at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers.

Show Notes

Transcript

John: With so many demands on faculty time, it can be difficult to prioritize professional development in the area of AI. In this episode, we examine a program that incentivizes faculty development in the AI space.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

John: Our guest today is Marc Watkins. Marc is an Academic Innovation Fellow at the University of Mississippi, where he helped found and currently directs the AI Institute for Teachers. Welcome back, Marc.

Marc: Thank you, John. Thank you, Rebecca. It’s great to be back.

Rebecca: We’re glad to have you. Today’s teas are:… Marc, are you drinking tea?

Marc: I am. I have a Cold Brew Hibiscus, which is really great. It’s still very warm down here in Mississippi. So it’s nice to have something that’s a little bit cool. That’d be refreshing.

Rebecca: That sounds yummy. How about you, John?

John: I am drinking a peppermint spearmint tarragon blend today. And it’s not so warm here. In fact, my furnace came on for the first time yesterday.

Rebecca: Yeah, transitions. And, I have English tea time today.

Marc: Well, that’s great.

John: So we have invited you here to discuss your ongoing work related to ChatGPT and other AI tools. Could you first describe what the AI Institute for Teachers is and its origins?

Marc: Sure, I think that when I was last a guest here in January of this year on your show. And it seems like 1000 years ago [LAUGHTER], but during that spring semester, I really took a much deeper dive than the original pilot with a lot of the generative AI tools in the fall. And we started noticing that the pace that big tech was deploying these tools and integrating these with existing software from Microsoft and Google was only accelerating. So in about April or May, I went to my chair, Stephen Monroe, and said, “I think we need to start training some people to get them prepared for the fall,” because we kind of thought that fall was going to be what it is right now, which is a chaotic just sort of mash up of sort of everything you can imagine that some people dive in deeply, some people tried to ban it, some people are trying to do some critical approaches with it too. So we actually worked with the Institute of Data Science here at the University of Mississippi, and we got some money. And we were able to pay 23 faculty members $1,000 apiece to train them for a day and a half about everything we knew about Generative AI, about AI literacy, ethics, what tools were working in the classroom, which wasn’t. And their whole goal was to go back to their home departments over the summer and serve as ambassadors and help prepare them for the fall semester. And we started that, we’ve had funding for one Institute, and now we’re doing workshops, and searching, as we all will, for more funding for doing,

Rebecca: How did faculty respond to (A) the incentive, but (B) also [LAUGHTER] the training that went with it?

Marc: Well, not surprisingly, they responded really well to the incentives, where you can pay people for their time, they generally do show up and do so as well. We had quite a few people wanting to take the training both internally from the University of Mississippi and then people started finding out about it, because I was posting it out on Twitter, and writing about it on my substack. So when we had interest from graduate students in Rome, interest from other SEC schools wanting to attend, and even interest from a community college in Hawaii. Definitely seen a lot of interest within our community, both locally and more broadly, nationally.

Rebecca: Did you find that faculty were already somewhat familiar with AI tools? I had an interesting conversation with some first-year students just the other day, and we were talking about AI and copyright. And I was just asking, “Hey, how many of you have used AI?” And I and another faculty member indicated that we had used AI to make it safe to indicate. And many of them really kind of shook their heads like “No, they hadn’t,” and they were unsure. And then I started pointing to places where we see snippets of it, in email and in texting and other places where there’s auto-finishing of sentences and that kind of thing. And then they’re like, “Oh, yeah, I have seen that. I have engaged with that. I have used that.” What did you find faculty’s knowledge?

Marc: Extremely limited. They thought of AI as ChatGPT. And one of the things we did with the session was basically frame it out as “Look, this was not just going to remain as a single interface anymore.” One of the things that actually happened during the institute that was completely wild to me was the last day, I woke up that morning. And I’d signed up through Google Labs, and you can do it as well, to turn on the features within the Google suite of tools, including in search and Google Docs, and Sheets and everything else. And they gave me access that last day, right before we began. And so I literally just plugged in my laptop and said, “This is what it’s going to look like in Google docs when you have generative AI activate in Google Docs. it pops up and immediately greets you with a wand with a phrase “Help me write.” And what I tried to explain to them and explained to faculty ever since then, is that it makes having a policy against AI very difficult when it shows up at an existing application with no indication whatsoever that this is in fact Generative AI. It’s just another feature that’s in the application that you have grown up with, from many of our students’ perspectives their entire lives. So yeah, we need to really work on training faculty, not just in the actual systems itself, but also getting them outside of that mindset that AI that we’re talking about is just ChatGPT. It’s a lot more than that.

John: Yeah, in general, when we’ve done workshops, we haven’t had a lot of faculty attendance partly because we haven’t paid people to participate [LAUGHTER], but what’s been surprising to me is how few faculty have actually explored the use of AI. My experience with first-year students was a little different than Rebecca, about half of the students in my large intro class had said that they had explored ChatGPT, or some other AI tool. And they seem pretty comfortable with it. But faculty, at least in our local experience, have generally been a bit avoidant of the whole issue. I think they’ve taken the approach that this is something we don’t want to know about, because it may disrupt how we teach in the future. How do you address that issue, and getting faculty to recognize that this is going to be a disruptive technology in terms of how we assess student learning and in terms of how students are going to be demonstrating their learning, and also using these tools for the rest of their lives in some way?

Marc: That’s a great question. We trained 23 people, I’ve also been holding workshops for faculty too, and again, the enthusiasm was a little bit different in those contexts, too. And I agree that faculty, I feel like they feel overwhelmed and maybe some of them want to ignore this and don’t actually want to deal with it, but it is here and it is being integrated at phenomenal rates in everything around us too. But if faculty don’t come to terms with us, and start thinking about engagement with their technology, both for themselves and for their students, then it is going to create incredible disruption that’s going to be lasting, it’s not going to go away. We’re also not going to have things like AI detection, like it is with plagiarism detection to come in and save the day for them too. And those are all things we’ve been trying to very carefully explain to faculty and get them on board. Some of them though, just aren’t there yet, I understand that. I empathize, too. This is a huge amount of time that you spend on these things to think about and talk about as well. And we’re just coming out of the pandemic, people are exhausted, they don’t want to deal with another, quote unquote, crisis, which is another thing that we’re seeing too. So there’s a lot of factors that are at play here that make faculty engagement, less than what I’d like to see.

Rebecca: We had a chairs’ workshop over the summer, and I was somewhat surprised based on our experience with other interactions with faculty, how many chairs had used AI. The number was actually a significant number. And most of them were familiar. And that to me was encouraging [LAUGHTER], it was like, “Okay, good, the leaders of the ship are aware. That’s good, that’s exciting.” But it’s also interesting to me that there are so many folks who are not that familiar, who haven’t experimented, but seem to have really strong policies around AI use or this idea of banning it or wanting to use detectors, and not really being familiar with what they can and cannot do.

Marc: Yeah, that’s very much what we’re seeing across the board too, is that the first detectors that I’m aware of that really came online, I think, for everyone was basically GPTZero, there are a few others that existed beforehand to IBM had one called the Giant Language Testing Lab. But those were all based on GPT-2, you’re going back in time to 2019. I know how ridiculous is it to go back four years in technology terms and think about this… that was a long time ago. And we really started adopting that through education or seem to be adopted in education based off of that panic. The problem is in incidents of education putting a system like that in place, it’s not necessarily very reliable. TurnItIn also adopted their own AI detector as well too. A lot of different universities began to explore and play around with it, I believe, and I don’t want to be misquoted here or misrepresent TurnItIn. I think what they initially came out with it, they were saying there was only 1% false positive rate for detecting AI. They’ve since raised that to 5%. And that has some really deep implications for teaching and learning. Most recently, Vanderbilt Center for Excellence in Teaching and Learning made the decision to not turn on the AI detection feature in TurnItIn. Their reasoning was that they had, I think, in 2022 some 75,000 student papers submitted. If they had the detector on during then that would give them a false positive grade about 3000 papers. And they just can’t deal with that sort of situation through a university level..No one can. You’d have to go through it investigating each one. You would also have to get students a hearing because that is part of the due process. It’s just too much. And that’s one of the main concerns that I have about the tools that it’s just not reliable in education.

John: And it’s not reliable both in terms of false positives and false negatives. So some of us are kind of troubled that we have allowed the Turnitin tool to be active and have urged that our campus shut it down for those very reasons, and I think a number of campuses, Vanderbilt was one of the biggest ones, I think to do that, but I think quite a few campuses are moving in that direction.

Marc: Yes, the University of Pittsburgh also made the decision to turn it off. I think several others did as well, too.

Rebecca: It’s interesting, if we don’t have a tool to measure, a tool to catch if you will, then you can’t really have a strong policy saying you can’t use it at all. [LAUGHTER] There’s no way to follow up on that or take action on that.

Marc: Where we’re at, I think, that for education, that’s a sort of conundrum. We’re trying to explain this to faculty. I think much more broadly, in society, though, if you can’t have a tool that works when you’re talking about Twitter, I’m sorry, X now, and understanding if the material is actually real or fake, that becomes a societal problem, too, and that’s what they’re trying to work on with watermarking. And I believe the big tech companies have agreed to watermark audio outputs, video outputs, and image outputs, but they’ve not agreed to do text outputs, because text is a little bit too fungible, you can go in and you can copy it, you can kind of change it around a little bit too much. So, definitely it’s gonna be a problem, too when state governments start to look at this, and they start wondering that the police officer taking your police report is writing this with their own words, the tax official using this as well, too. So it’s gonna be a problem well outside of education.

Rebecca: And if we’re not really preparing our students for that world in which they will likely be using AI in their professional fields, then we’re not necessarily doing our jobs and education and preparing our society for the future.

Marc: Yeah, I think training is the best way to go forward too and again, going back to the idea of intentional engagement with the technology and giving the students these situations where they can use it and where you, hopefully if you’re a faculty member, you actually have the knowledge and the actual resources to begin to integrate these tools and talk about the ethical use case, understanding what the limitations are and the fact that it is going to hallucinate and make things up, and to think about what sort of parameters you want to put on your own usage too.

John: One of the things that came out within the last week or so, I believe,… we’re recording this in late September… was the introduction of AI tools into Blackboard Ultra. Could you talk a little bit about that?

Marc: Oh boy, yes indeed, they announced last week that the tools were available to us in Blackboard Ultra. They turned it on for us here at the University of Mississippi, and I’ve been playing around with it, and it is a little bit problematic, because for right now, what you can do is with a single click, it will scan your existing materials in your Ultra course and it will create learning modules. It will create quiz questions based off that material, it will create rubrics, and will also generate images. Now compared to what we’ve been dealing with ChatGPT and all these other capabilities, this is almost a little milquetoast by comparison. But it’s also an inflection event for us in education, because it’s now here, it’s directly in our learning management system, it’s going to be something we’re going to have to contend with every single time we open up to create an assignment, or to do an assessment. And I’ve played around with it. It’s an older version of GPT. The image version I think is based on Dall-E, so you would ask for a picture of college students and you get some people with 14 fingers and weird artifacts all over their face, which may not be the one that would actually be helpful for your students. And while the other learning modules there are not my thinking necessarily, it’s just what the algorithm is predicting based off the content that exists in my course. We have that discussion with our faculty, we have them cross that Rubicon on and saying, “Okay, I’m worried about my students using this, what happens to me and my teaching, my labor, if I start adopting these tools. There could be some help, definitely, this could really streamline the process, of course creation and actually making it aligned with the learning outcomes my department wants for this particular class.” But it also gets us in a situation where automation is now part of our teaching. And we really haven’t thought about that. We haven’t really gotten to that sort of conversation yet.

Rebecca: It does certainly raise questions about, obviously, many ethical questions and really about disclosing to students what has been produced by us as instructors and what has been produced by AI and authorship of what’s there. Especially if we’re expecting students to [LAUGHTER] do the same thing.

Marc: It is mind boggling, the cognitive dissonance, with having a policy and saying “No AI in my class,” then all of a sudden, it’s there in my Blackboard course, and I could click on something. And, at least at this integration of Blackboard, they may very well change this, but once you do this, there’s no way to natively indicate that this was generated by AI. You have to manually go in there and say this was created. And I value my relationship with my students, it’s based off of mutual trust. I think almost everyone in education does. If we want our students to act ethically, and use this technology openly, we should expect ourselves to do the same. And if we get into a situation where I’m generating content for my students and then telling [LAUGHTER] them that they can’t do the same with their own essays, it is just going to be kind of a big mess.

John: So given the existence of AI tools, what should we do in terms of assessing student learning? How can we assess the work reasonably given the tools that are available to them?

Rebecca: Do you mean we can just use that auto-generated rubric right, that we just learned about? [LAUGHTER]

Marc: You could, you can use the auto-generated rubric separately from Blackboard. One of the tools I’m piloting right now is the feedback assistant, it was developed by Eric Kean and Anna Mills. I consulted with them on this, too. She’s very big on the AI space for composition. It’s called MyEssayFeedback. And I’ve been piloting this with my students. They know it’s an AI, they understand this. I did get IRB approval to do so. But I’ve just got the second round of generated feedback, and it’s thorough, it’s quick, it’s to the point. And it’s literally making me say, “How am I going to compete with that?” And maybe the way is that maybe I shouldn’t be competing with that, maybe it’s I’m not going to be providing that feedback. But then maybe then I should be providing my time in different ways. Maybe I should be meeting with them one on one to talk about their experiences, maybe that way. But I think you raise an interesting question. I don’t want to be alarmist, I want to be as level-headed as I can. But from my perspective, all the pieces are now there to automate learning to some degree. They haven’t been all hooked up yet and put together a cohesive package. But they’re all there in different areas. And we need to be paying attention to this.Our hackles need to be raised just slightly at this point to see what this can do. Because I think that is where we are headed with integrating these tools into our daily practice.

Rebecca: AI generally has raised questions about intellectual property rights. And if our learning management systems are using our content in ways that we aren’t expecting, how is that violating our rights or the rights that the institution has over the content that’s already there.

Marc: A lot of perspectives of the people that I speak with too, their course content, their syllabi, from their perspective is their own intellectual property in some ways. We get debates about that, about the actual university owns some of the material. But we have had instances where lectures were copyrighted before in the past. And if you’re allowing the system to scan your lecture, you are exposing that to Generative AI. And that gets at one aspect of this. The other aspect, which I think Rebecca is referring to is the issue with training this material for these large language models itself could indicate that it was stolen or not properly sourced from internet and you’re using it and then you’re trying to teach your students [LAUGHTER] to cite material correctly too, so it’s just a gigantic conundrum of just legal and ethical challenges. The one silver lining in all this, and this has been across the board with everyone in my department. This has been wonderful material to talk about with your students, they are actually actively engaged with it, they want to know about this, they want to talk about it. They are shocked and surprised about all the depths that have gone into the training of these models, and the different ethical situations with data and all of it too. And so if you want to just engage your students by talking to them about AI too, that’s a great first step in developing their AI literacy. And it doesn’t matter what you’re teaching, it could be a history course, it could be a course in biology, this tool will have an impact in some way shape or form in your students’ lives they want to talk about, I think maybe something to talk about is there are a lot of tools outside of ChatGPT, and a lot of different interfaces as well, too. I don’t know if I talked about this before in the spring, the one tool that’s really been effective for a lot of students were the reading assistant tools, one that we’ve been employing is called ExplainPaper. They upload a PDF to it, it calls upon generative AI to scan the paper and you can actually select it to whatever reading level you want, then translate that into your reading level. The one problem is that students don’t realize that they might be giving up some close reading, critical reading skills to it as well too, just like we do with any sort of relationship with generative AI. There is kind of that handoff and offloading of that thinking, but for the most part, they have loved that and that’s helped them engage with some really critical art texts that normally would not be at their reading level that I would usually not assign to certain students. So those are helpful. There are plenty of new tools coming out too. One of them is called Claude 2 to be precise by Anthropic. That just came out, I think, in July for public release, it is as powerful as GPT-4. It is free right now, if you want to sign up for it as well too. The reason why I mentioned Claude is that the context window, what you can actually upload to it is so much bigger than ChatGPTs. I believe their context window is 75,000 words. So you can actually upload four or five documents at a time, synthesize those documents. One of the things I was using it for as a use case was that I collected tons of reflections for my students this past year about the use of AI. It’s all in a messy Word document. It’s 51 pages single spaced. It’s all anonymized so there’s new data that identifies them. But it’s so much of a time suck on my time, just go through to code those reflections. And I’ve just been uploading to Claude and having it use a sentiment analysis to point out what reflections are positive from these students, in what way, and it does it within a few seconds. It’s amazing.

John: One other nice thing about Claude is that has a training database that ends in early 2023. So it has much more current information, which actually, in some ways is a little concerning for those faculty who were trying to ask more recent questions, particularly in online asynchronous courses, so that ChatGPT could not address those. But with Claude’s expanded training database, that’s no longer quite the case.

Marc: That’s absolutely correct. And to add to this rather early discussion about AI detection, none of the AI detectors that I’m aware of had time to actually train on Claude, so if you generated essay… and you guys are free to do this on your own, your listeners are too… if you generated and essay with Claude, and you try to upload that to one of the AI detectors, very likely you’re going to get zero detection or a very low detection rate for it too, because it’s again, a different system. It’s new, the existing AI detectors have not had time. So the way to translate this is don’t tell your students about it right now, or in this case, be very careful about how you introduce this technology to your students, which we should do anyway. But this is one of those tools that is massively popular, a lot of people just haven’t known about it because, again, ChatGPT just takes up all the oxygen in the room when we talk about Generative AI

John: What are some activities where we can have students productively use AI to assist their learning or as part of their educational process?

Marc: That’s a great question. We actually started developing very specific activities for them to look at different pain points for writing classes. One of them was getting them to actually integrate the technology that way. So we built a very careful assignment, which called on very specific moves for them to make both in terms of their writing, and their integration of the technology for that. We also looked at bringing some research question, building assignments that way. We have assignments from my Digital Media Studies students right now about how they can use it to create infographics. Using the paid for version of ChatGPT Plus, they can have access to plugins, and those plugins then give them access to Canva and Wikipedia. So they can actually use Canva to create full on presentations based off of their own natural language and use actual real sources by using those two plugins in conjunction with each other. I just make them then go through it, edit it with their own words, their own language too, and reflect on what this has done to their process. So lots of different examples, too, I mean, it really is limited only to your imagination in this time, which is exciting, but it’s also kind of the problem that we’re dealing with, there’s so much to think about.

Rebecca: From your experience in training faculty, what are some getting started moves that faculty can take to get familiar enough to take this step of integrating AI by the spring?

Marc: Well, I think the one thing that they could do is, there are a few really fast courses. I think it’s Ethan Mollick from even from the Wharton School of Business put out a very effective training course that was all through YouTube, I think it’s like four or five videos, very simple to take, to get used to understanding how ChatGPT works, how Microsoft’s Bing works as well too, and what sort of activities students can use it for, what sort of activities faculty could. Microsoft has also put out a very fast course, I think takes 53 minutes to complete about using generative AI technologies in education. And those are all very fast ways of basically coming up to speed with the actual technology.

John: And Coursera has a MOOC through Vanderbilt University, on Prompt Engineering for ChatGPT, which can also help familiarize faculty with the capabilities of at least ChatGPT. We’ll include links to these in the show notes.

Marc: I really, really hope Microsoft, Google and the rest of them calm down, because this has gotten a little bit out of control. And integration of these tools are often without use cases, they’re often waiting to see how we’re going to come up and use them too. And that is concerning. Google has announced that they are committed to releasing their own model that’s going to be in competition with GPT4, I think it’s called Gemini by late November. So it looks like they’re just going to keep on heating up this arms race and you get bigger models, more capable and I think we do need to ask ourselves more broadly what our capacity is just to keep up with this. My capacity is about negative zero at this point… going down further.

John: Yeah, we’re seeing new AI tools coming out almost every week or so now in one form or another. And it is getting difficult to keep up with. I believe Apple is also planning to release an AI product.

Marc: They are. They also have a car they’re planning to release, which is the weirdest thing in the world to me, that there could be your iPhone charged in your Apple Car.

John: GM has announced that they are not going to be supporting either Android or Apple CarPlay for their electric vehicles. So perhaps this is Apple’s way of getting back at them for that. And we always end with the question, what [LAUGHTER] is next, which is perhaps a little redundant, but we do always end with that.

Marc: Yeah, I think what’s next is trying to critically engage the technology and explore it not out of fear, but out of a sense of wonder. I hope we can continue to do that. I do think we are seeing a lot of people starting to dig in. And they’re digging in real deep. So I’m trying to be as empathetic as I can be for those that don’t want to deal with the technology. But it is here and you are going to have to sit down and spend some time with it for sure.

John: One thing I’ve noticed that in working with faculty, they’re very concerned about the impact of AI tools on their students and student work. But they’re really excited about all the possibilities that opens up for them in terms of simplifying their workflows. So that, I think, is a positive sign.

Rebecca: They could channel that to help understand how to work with students.

Marc: I hope they find that out, there’s a positive pathway forward with that too.

John: Well, thank you. It’s great talking to you and you’ve given us lots more to think about.

Marc: Thank you guys so much.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]

308. Design for Learning

We tend to design courses for ourselves because we are the audience we know best. In this episode Jenae Cohn joins us to explore how user-experience design principles can help us create effective and engaging learning experiences for the students we have right now. Jenae is the Executive Director of the Center for Teaching and Learning at the University of California at Berkeley. She is the author of Skim, Dive, and Surface: Teaching Digital Reading. Her newest book, co-authored with Michael Greer, is Design for Learning: User Experience in Online Teaching and Learning.

Show Notes

  • Cohn, J. (2021). Skim, dive, surface: Teaching digital reading. West Virginia University Press.
  • Cohn, J., & Greer, M. (2023). Design for learning: User Experience in Online Teaching and Learning. Rosenfeld Media
  • Global Society of Online Literacy Educators
  • Horton, S., & Quesenbery, W. (2014). A web for everyone: Designing accessible user experiences. Rosenfeld Media.
  • Web Accessibility Guidelines
  • Copies of Design for Learning may be ordered at the Rosenfeld Media website. The discount code for listeners is TEA20. It’ll be available on Wednesday, 9/27 and will give listeners access to 20% off the book for one month (i.e. 30 days).

Transcript

John: We tend to design courses for ourselves because we are the audience we know best. In this episode we explore how user-experience design principles can help us create effective and engaging learning experiences for the students we have right now.

[MUSIC]

John: Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.

Rebecca: This podcast series is hosted by John Kane, an economist…

John: …and Rebecca Mushtare, a graphic designer…

Rebecca: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners.

[MUSIC]

Rebecca: Our guest today is Jenae Cohn. Jenae is the Executive Director of the Center for Teaching and Learning at the University of California at Berkeley. She is the author of Skim, Dive, and Surface: Teaching Digital Reading. Her newest book, co-authored with Michael Greer, is Design for Learning: User Experience in Online Teaching and Learning. Welcome back, Janae.

Jenae: Thank you. I’m so glad to be back.

John: It’s good to see you again.

Jenae: …Good to see you, too.

John: Today’s teas are… Jenae, are you drinking any tea?

Jenae: I sure am. I’m always prepared to drink tea. Especially when I’m talking to the two of you. But I went for a classic English breakfast tea this morning. Do you both have some tea with you?

Rebecca: Yeah, I have English tea time.

Jenae: We’re matching…

Rebecca: Yeah…

[LAUGHTER]

John: And, I’m not. I have [LAUGHTER] ginger peach black tea today.

Jenae: That sounds really good, though.

John: It is.

Rebecca: Sounds like a good way to start the day, for sure. So we invited you here today to discuss Design for Learning. Can you talk a little bit about how this book project came about?

Jenae: Absolutely. So my colleague Michael and I have a lot of shared interests. Michael and I both are trained in rhetoric and composition. And we both are people really interested in online writing, online reading, and online learning, broadly speaking. We both served on the board for the Global Society of Online Literacy Educators, which is an organization dedicated to supporting folks who teach reading and writing online, broadly speaking. Through that organization, we got to know each other better. And we just realized how much we wanted to talk about what it really meant to create quality online learning experiences. And something that kept cropping up for the two of us. And I should say that both of us have had like a hodgepodge of jobs in and around higher education. We kind of joke that we were both sort of like these misfits in higher ed, people who have kind of done a bit of teaching, a bit of admin. He’s worked in publishing, I did a lot of work in instructional design and just higher education pedagogy. And something we noticed, just in the various roles that we were in was that educators, professors, faculty could learn a lot from user experience frameworks. And we were reading a lot about UX and UI in the work that we were doing around instructional design and for him publishing, and it just dawned on us like, why are we not bridging these conversations between the work of thinking about designing learning interfaces and the work of building really good, high quality learning experiences. I think we notice that in higher ed, there is this tendency to kind of try and reinvent the wheel around defining what a good teaching experience, especially what a good online teaching experience is by just creating really kind of exhausting templates and tons of checklists and rules. And we really thought those are useful, but wouldn’t it be more useful just to remember that students are people navigating devices online? And can’t we use the frameworks that help inform those design decisions to inform the design of learning experiences to make them better? So that was really the genesis of this project. We started off thinking we’d write a bunch of blog posts and then it struck us that blogs and articles were great, but wouldn’t it be even better if we wrote a book [LAUGHTER]. So we put it all together, and it resulted in this book.

John: So who’s the intended audience of this book?

Jenae: We really are targeting a broad audience with this book, I’d say primarily folks who do instructional design style work in mind. So in higher ed, that could be faculty, a lot of faculty play the role of instructional designers, as well as facilitators and teachers, of course. But, we also hope that this book would really reach folks who do dedicated instructional design support. We also hope that this would just reach people who are having to teach online or do trainings or workshops online, and who are still really struggling with it. This book, I would say, was written before the pandemic happened. We were, I would say, drafting and conceptualizing it before the pandemic. And of course, the pandemic shaped the drafting as we went, there’s still COVID-19 out there, so I don’t want to say we’re beyond the pandemic. But in this moment where we’re beyond perhaps like a peak point of the pandemic, let’s just say. There may be folks who are still wanting to be more intentional about what it means to provide more equitable access to online learning experiences, who want to be designing in a more intentional way, and who want to be really thinking critically about how to create more sustainable online learning experiences, as well, that really work. I think we were also on a mission with this book to prove that really, anyone can do this, you just need to keep some known principles in mind that, again, this is not totally new territory, and scholars and user experience and human computer interaction have been thinking for a very long time about how to make information accessible online, and how to make sure that information and interactions are easily navigable. And so that was really the literature we wanted to tap into. So that’s all to say that I think people who benefit from reading this book is really anyone who wants to be creating a better online learning experience for whatever teaching situation they’re in.

Rebecca: I’m, of course, super excited about this book, because I’m a UX designer. I love that you use that framework to write this book. Can you talk a little bit about why you chose this approach?

Jenae: Absolutely. I’m so glad that you appreciate this book exists. We’ve gotten really good reception from the UX community on it as well. I would say that we use this framework because we felt like it really centered the learner in an important kind of way. I think that in a lot of teaching situations, people who educate or design learning are often more thinking about the content: What information do I have to deliver? What are the main things that I need to make sure people know how to do? Those aren’t bad things to focus on, we need to cover content, and we need to make sure that there are clear outcomes. But I think it’s most important to really think about how is someone engaging with that content? How are they understanding it? What are their opportunities to understand that content in a variety of different ways. And I think what a user experience framework allows us to do is to center that reminder. Learners have these embodied experiences that shape how well they’re going to be able to learn, how well they’re going to be able to interface with the information. And if we’re talking about that in an online context, in particular, it’s impossible to do so without addressing what it means to, again, engage with and use these online environments effectively. So I think a UX framework really just allows us to be more centered in reminding ourselves who really benefits from the learning experiences we design, and who really needs to have access to [LAUGHTER] the information to be successful. And I think UX frameworks just really help us center that.

John: Can you talk a little bit about how this approach centers the user in terms of practical ways in which that’s built into the design process?

Jenae: Sure, one way to sort of think about that is to really take a step back and try to remind yourself just who is taking your class in the first place. Starting there, starting from the place of trying to be curious about who your learners really are. I think that it’s easy to make assumptions, I’ll just say in higher education, in particular, since I think that’s primarily the audience for this particular podcast. I think a common misconception, for example, is that all students entering their class are traditional college age, 18 to 21 years old, but like, I should put a big asterisk on that and say, that’s probably not the traditional age at most institutions anymore. But that’s the stereotype of kind of who a college student is. And there may be some assumptions about what their prior learning experiences were like that brought them into a college classroom… about the prior knowledge that they had. And so what I think a user-centered design encourages us to say, “Do we know that? How do we know that? What information do we need to gather to remember who’s actually coming into our rooms?” And I’m not suggesting that any educator has to like, do deep dive demographic data work to find out who their learners are. But I think most of us can kind of anticipate the range of people who are coming into our classes. We might anticipate just the different types of learners that we may encounter. And by that, I mean, it’s worth, I think, before you start designing, just trying to remember, what are the different motivations that students have for coming into the class? What are their purposes for being there? What are the main things that students are going to want to do by being in your class or your training or your workshop at any given moment? So starting by just sort of trying to map out who those people are? And then try to anticipate, okay, given this motivation, or this purpose that this learner may have, what kinds of things might they be looking for… literally looking for my online course? What things will they click on first? Which links are they going to want to access most frequently? Which resources are going to benefit them on the site most? And then trying to design your learning management system course site, or if you’re not using learning management system, your course website, broadly speaking, to really privilege the resources, the links, the activities, the pages that are going to be best aligned with what you anticipate your users or your learners may need. And Rebecca, I’m sure, can speak to this given her expertise, too, but UX design really is a whole process of trying to consider how the visual information, how even like the tactile information, say how your keyboard was set up, how your device is set up, how that allows you to most easily use and engage with products, so to speak, that you’re building. And in this case, we want to think about how can you build the best online course that you can, in a way that allows users to most easily find the information you anticipate they will most frequently need?

Rebecca: So one of the things I’m hearing you say, is really thinking about the wide variety of learners that we have and the different needs that they have and trying to address that. One of the things that’s really popular in UX design and that you talk about in your book are personas. Can you talk a little bit about how learner personas can help us think through the different kinds of learners that we have in our class in a really practical, tangible way. You just kind of provided that theoretical framework, but I love that the personas is such a practical application of that.

Jenae: Yes, thanks for asking that. Rebecca, I was debating whether to dive into that with the last question. But let’s dive into it now. So for those who aren’t familiar, personas are an exercise where you really try to create a character sketch, I would say, of the user you’re imagining is going to engage with your course or in this case, try to imagine an example of a student who’s going to be in your class. And by creating a character sketch, I mean, I encourage instructors, if they have the time to sit down and say, “Okay, what might be the name of someone in my class? What might be their age? What might their prior experiences with learning my topic have been? Why are they here? What brings them to college? Or what brings them to this class in the first place? What are going to be some of their biggest challenges? What are going to be some of their biggest hopes? What are the things that they’re going to be most excited about doing in this class?” And again, it’s a bit of an imaginative exercise. And so I think it’s easier to do with more teaching experience. But it’s also not impossible to do even if you’ve had relatively limited experience. It’s really just an exercise in trying to think through who might be the real people that you are engaging with, I do want to say that there’s been a lot of conversation in the UX community, and again, Rebecca, you may have some thoughts on this, too, about sometimes the stereotypes that personas can perpetuate. For example, I think there have been concerns in the UX community that when you try to characterize, say, an older user, of an online interface, a stereotype might be that they struggle more or are more challenged with using technology than, say, a younger user. And that that might be a challenge to anticipate. And so I want to be mindful, for example, that if you are going to be in the practice of building personas, which we talk about in the book, because I do think is a useful exercise to kind of try and make concrete for yourself who is going to be on the receiving end of your experience, that you do try to check yourself a little bit on reinforcing stereotypes to the best of your ability. It’s easy to do, stereotypes exist because we notice patterns sometimes in how people behave. And that can sort of reproduce some harmful assumptions about who those users are. But again, to the best of your ability, attempt to anticipate what the needs might be based on what you do know about who might be in the room, just again, kind of reminding yourself that you’ll want to think about your personas in nuanced ways, and not necessarily make assumptions about who they are. And I would say one solution to that, how am I supposed to write a generalized description of a persona, while avoiding all possible stereotypes about who they might be? I would say again, time allowing, try to run your personas by other people, and just see what their reactions are to reading them. For example, if you have a trusted colleague, or a friend who teaches a similar class, or who you work with regularly, just show them what you’ve drafted and say, “Does this feels like a real person to you?” And to attempt to ask diverse people about how your persona sketches are landing or how realistic they feel to them. That’s always a good way to kind of gut check, and just make sure that as you’re anticipating your users’ needs, you’re not falling too much into your own biases about who the people are that you’re supporting in your course.

Rebecca: So one of the things, I think, people do sometimes run into when they’re making personas is to create the ideal student that doesn’t exist, and also to recreate themselves. And so one strategy that I often recommend is thinking about creating aggregates of people that you do know. Because then they’re more realistic in terms of the way they might interact. So if you’ve taught a class before, you might have a real pool of people you could draw from [LAUGHTER] and to create a persona from, obviously, that’s more difficult when it’s a new place. And I was also going to offer up in terms of thinking about disability and thinking about accessibility, that there’s a book called A Web for Everyone, and they have a lot of resources. It was published quite a while ago, but they have a lot of resources still online, they have some personas for people with a wide range of different kinds of disabilities. And sometimes that can be really useful in just thinking through kinds of scenarios that you might not think of on your own.

Jenae: That’s fabulous. I would love to see that resource about sort of supporting accessibility, especially. That’s such a huge issue in designing online learning experiences, particularly. I’m so glad you mentioned those resources. That’s fantastic.

John: And while there may be those types of biases that you might have, those who’ve taught classes multiple times do know some of the types of problems that past students have had. So those issues that they’ve experienced in the past could be built in. But one of the other things you suggest is doing a pre-course survey, so that you get some more information about the actual students in the room rather than those who may have been thinking about when you initially designed the course? Could you talk a little bit about that survey?

Jenae: Yes, I’d be happy to talk about the pre-course surveys. So this is a practice that, I think, has multiple benefits. So in a pre-course survey, I think instructors have this wonderful opportunity just to ask students what their motivations are for engaging with the class, what brought them here, how they would characterize some of their prior experiences with learning similar topics, if any, and just to voice what concerns they have, or what things are exciting to them about the term ahead. I’m giving a lot of examples of possible questions and I just want to acknowledge that not all instructors will want to ask all of those questions all at once. But those kinds of questions that really get at motivation and concerns, I would say, in a nutshell, can be really critical, both for adjusting, I think, those persona expectations. So, creating personas should be an iterative process, I should say, as well. It’s not a one and done thing where you anticipate who your learners are prior to the course starting and then you’re like, “Okay, I figured it out, I know who all the students are. Knowing who the real students are, can then allow you to go back to what you anticipated. I think, and both of you, Rebecca and John, were speaking to how you could use prior information from prior terms to inform your kind of current term or current course. Great, you could sort of just align your prior understanding with this current information you might get from these surveys to then go into your course website, or your course learning management system, your syllabus and say, “Okay, is this design going to work for the group of people who are actually here based on what I’m reading?” …recognizing, of course, that nothing’s gonna be perfect for everyone. But you can do the best you can to try and make the materials as good as possible for the group that you have. In front of you. I would say that you want the survey to feel less burdensome for your students to complete. I’m giving a lot of examples of questions that I think are ideal as open-ended questions. Some of these, you could turn into multiple choice or kind of Likert-scale style questions, because you can just use it as an opportunity to take the temperature. “On a scale of one to 10, for example, how confident do you feel in your understanding of your ability to pick up new quantitative concepts?” …for example, if you’re teaching in a STEM-style discipline. Or “On a scale of one to 10, how comfortable do you feel as a writer or with writing tasks?” …if you’re teaching something more humanities- or writing-centric. You can get really creative in trying to solicit some feedback. And I also encourage instructors to be judicious in what they’re asking in these pre-course surveys to kind of try and ask questions, with the end goal of helping you as the instructor make small tweaks to the design of the course. Think about this information as a way to say “Okay, are there certain links I should put on the homepage that I didn’t think needed to be on the homepage? Or should I reorganize the menu on my learning management system in a way that highlights some resources more than others based on the information I’m getting in the survey? Should I reorganize a module to introduce some content before other content, because I’m seeing a trend in the surveys about less confidence in one area of the course than I was expecting in another?” So thinking about how the answers might inform your design, a research-based perspective really, I think, can make your course really even stronger. And I think it’ll feel better, both for you and the students because it helps the students see that you’re curious about them, you want to know who they really are. And we know that engaging personally with people really matters for good teaching. But the instructor too, it can be really frustrating. If you design something and it doesn’t land with your students. You feel like you spent a lot of time building something that didn’t work. That’s a really disheartening experience. So getting the feedback might allow you to avoid [LAUGHTER] having or feeling so disappointed if the information didn’t land the way you were expecting it to. And this isn’t foolproof. There’s always room, again, for iteration. But I do think the surveys can at least help you anticipate a little bit better how the progression through your course could go.

Rebecca: I can imagine that some of those surveys with open-ended questions could lead to better understanding how students name things or label things which could give you a lot of clues about the actual user design of a course by just how you might name or provide quick descriptions of things. In your book, you talk a lot about instructional text design, which obviously has lots of skills in online learning from instructions for assignments to just how we might label a folder [LAUGHTER]. There’s lots of skill there. Can you talk a little bit about the basic principles that you’d recommend for course designers to follow when they’re writing instructional text?

Jenae: Absolutely, and I realized, as you were talking and responding, I was nodding along. And then it struck me. It’s like, “I’m on a podcast, no one’s going to know that I’m nodding and agreeing with you right now.” So [LAUGHTER] for the listeners sake, like I was nodding along quite vigorously with that entire response. Instructional text, I think, is one of the most underrated and one of the most important things to design for any online course experience. I think that online course designers have a real tendency to rely too heavily on video and on images. There’s an assumption that if you’re working online, everyone’s just using video all the time, or everyone’s just wanting to engage with the flashiest multimedia possible. That is still important. I mean, we have two chapters in the book, all dedicated to video. So I don’t want to undermine that. It is important to engage with multimodal artifacts and building multimodal interventions, when you’re teaching in a multimodal environment like the internet. However, for students who may have low internet access and low bandwidth, for students with disabilities, text remains one of the most accessible and easiest ways to find information in an online course. I’d also say text is one of the most mobile-friendly pieces to think about. And we know that increasing number of students are accessing their courses or coursework through their smartphones. I’ll answer your question directly now, but I wanted to provide that context. I would say when it comes to designing instructional text, I really encourage instructors to think about two big things, to think about the hierarchy of the information that they’re writing, and to think about the discrete chunks of information that they’re wanting to communicate. So when I talk about the hierarchy of texts, I think it’s important when we’re writing to consider what are the sections of our text? Most academics, most instructors, are used to, when they’re reading or writing, creating headers, and sub-headers, and paragraphs that denote a certain order of information. And when you’re teaching online, especially, I think even more critically about how are you labeling the text? How are you indicating which things are instructions versus content? How are you labeling the order of the content that you want students to read in? How are you even labeling the order of instructions, like there is usually multi-tiered sets of steps. So using header text and different layers of header text, is a really important web accessibility measure. And again, it helps readers see visually and if they’re using a screen reader tool, it helps them navigate that text more easily. So I should take one step back and say when I’m referring to header text, I mean that when you’re working in a rich text editor, on any website, you can typically see an option to select different layers of headers, like the header ones are usually the highest, biggest level header, header twos go below that header, threes go below that. So just being mindful that just increasing text size is not the same thing as using headers is one really, really simple way to create hierarchy. And again, to denote the correct order of reading the text information. And when I say chunking text, this is as simple as just thinking about paragraphing, making sure that you are spacing out pieces of content in really critical ways. So anyone who’s read a piece of writing with super long paragraph knows, that’s a lot harder to kind of discern, it’s a lot harder to see how one idea moves from one to the next. Shorter paragraphs are typically easier to get a sense of when you’re moving from one idea to a new idea. And so even though long paragraphs have their purpose, perhaps especially in scholarly writing, or even in more, I would say kind of creative writing, in some cases, when you’re doing really instructional or technical work, which you’re often doing when you’re designing a class, shorter is better, more chunked is easier to access, because you’re assuming that people are doing things with your information. So those are the two qualities I would just be thinking about with instructional text. There’s a whole other component that we didn’t really address in the book, but I’ll just stick to very briefly here, which is also thinking about just the visual appearance of your text. A lot of accessibility folks speak to some best practices and guidelines around font face, and font size, and some of these factors when you’re designing text as well. I’m not an expert, I should say, in like type of graphic design or font size, but I want to point out anyway, because I think if you are designing online, it’s important again to do the best that you can to try and anticipate those needs. So I think as a general rule, making sure your font sizes are not super teeny tiny, or super large. Making sure that you’re using standard font faces: Arial, Helvetica any sort of sans serif font is typically considered a best practice. The rules around this change all the time, Web Accessibility Guidelines change as technology evolves, so I never like to give super hard and fast rules, and again, it’s not my area of expertise. But it’s another piece to keep in mind that visual and verbal information is intertwined. Text is a visual medium, online learning experiences are largely a visual medium, by default. And so the more mindful we can be of what that looks like, and the more mindful we can be of how the visual experiences we design online, are compatible with accommodations for disabled users. We just anticipate our users’ needs, our learners’ needs more proactively, and it raises the boats for everyone. It just gives everybody a deeper chance to succeed if we’re just thinking about these interface choices in more deliberate ways.

Rebecca: I love that you’re really talking about how the instructional text is also part of digital accessibility. It’s important to have plain language, it’s important to chunk your content and these sorts of things. So I’m really excited that you’re incorporating that into the work that you’re doing.

Jenae: Thank you. It is exciting. I think it’s one of these things that, when Michael and I were first discussing this book, it was a real lightbulb moment for us that there was such a robust literature out there that discussed all these great principles for making sure that online information was easy to find. And it just was striking to us that a lot of folks in teaching professions weren’t getting access to that information or exposure to that information. And we started thinking about this, again, prior to the pandemic, kind of in the mid 2010s. And even at that point, online courses were growing, mobile access was becoming a more common way that students were engaging with courses. So, why not tap into these existing sets of conversations that are industry best practices, for engaging with online interfaces, in spaces like higher ed, and in spaces just like learning and development, where these dialogues seem not to have met each other as fully as they could.

Rebecca: Our chief technology officer and I were having a conversation about some of these things yesterday as we’re talking about our student body is diversifying and that we have far more students with disabilities who are able to attend college and have access to college in a way that maybe they haven’t in the past. And as you were talking about headings and paragraphs and things, something that people might not know, is that if you use a screen reader, you’re not necessarily visually interacting with the text. Instead, you’re thinking programmatically, and so just like kind of vision centered [LAUGHTER], the user might skim headings visually, it’s the same way someone might use a screen reader. So by choosing a heading level two, it allows someone to find that section easier. And by breaking things into paragraphs, and delineating that’s that kind of content that allows a screen reader user to be able to jump to a particular part of the content. When we don’t do that, a screen reader user has to listen to everything from the top to the bottom of the page.

Jenae: Great example. Yes, and that’s such a frustrating experience to have to do that. If we can be just a little bit more attentive to the information architecture of sort of what we’re trying to communicate and convey… information architecture is a technical term, but it’s also a metaphor [LAUGHTER]… we have architecture and we have design to help create solid foundations for places that we live. Similarly, when it comes to information, we need to be building solid infrastructure to help people navigate their way through a course. One of my colleagues a while ago used a metaphor for online learning design I’ve never forgotten and we’ve alluded this a bit in the book, which is that when you’re building something online, it’s like you’re just building a whole house [LAUGHTER]. When you walk into an in-person classroom, the architecture is literally there, and you make assumptions about the room in the space, the second that you walk in the door. When you’re designing text online, or just when you’re thinking about the whole online learning experience, it’s a total blank canvas, you have to build that architecture and those hierarchies. If you’re not attentive, you’re absolutely right, the consequence is that it can be a big overwhelming mess of information. And I think it’s a useful practice for instructors, even when they’re not teaching online, to think about these things. It’s also just a great exercise and getting really very focused on what information do you want to prioritize when you’re communicating assignment instructions or when you’re picking out content-based readings for your students? What do you want them to focus on? What are the big things you really need them to learn or pay attention to? And so if your course design, your visual design can align with the hierarchy of choices you’re making as an instructor or the priorities that you’re setting, it just makes it easier for everyone to have equal access that information so that more time can be spent for students to focus individually on how they’re processing, applying, doing higher-order thinking with that information. They don’t have to spend so much energy just trying to intake the kind of basics before they have the opportunity to really work with it and apply it meaningfully,

John: You provide a lot of other information in your book, and we encourage people to read your book. If they want to find out more about creating videos, about providing effective webinars, and so forth, there’s some really nice hints and suggestions throughout. But one of the things you end with in there, is ways in which instructors can continuously improve their courses, in terms of soliciting feedback to make the course better each time. Could you talk a little bit about how you would encourage instructors to continuously work on developing their courses?

Jenae: Sure. So I really like that section of the book, because what I hope that section communicates is that thinking about your course design is a reflective and an iterative process. I don’t think a course is ever really fully perfect and done, there’s always things you can do, and modify each time you teach or offer the experience. So, I don’t think getting feedback on the course has to be hard, I don’t think it has to take a ton of time. We talk about multiple ways of getting information about how the course is working. And I’m going to start with I think some of like the easiest and most passive ways to get information and then we’ll sort of work our way to some of the more perhaps active or personalized interventions for getting information about the course. So one thing I think is worth really paying attention to, after you finish teaching a course, are some of the analytics that are available in your learning management system or your course website. And I recognize that some folks are really reluctant to look at the analytics, because there is a surveillance economy implicated in the tracking of course analytics. Every site on the web tracks your movements, every site in the web knows how long you’ve stayed on a certain page, what things you’ve clicked on. And a learning management system is no exception to that. Unfortunately, that information can get weaponized to discriminate against students, discriminate against users in problematic ways. In the web, outside of learning, for example, analytics can be gathered and sold to advertising companies to spread information about your activities for profit. So I just want to note that context, but you can also use this information for good and for some useful things as well. So seeing which resources students are clicking on the most in your class can be really useful information for you to say, “Huh, seems like a lot of people found that resource useful.” You don’t have to necessarily identify which individual students looked at which resources but you can look at this data, typically in aggregate, and again, most learning management systems have an analytics dashboard, you can access to look at this. I think that’s incredibly useful just to see what was clicked most often and what wasn’t. You might also want to track, for example, which pieces of information students did spend more time on. It could indicate a couple different things, it might indicate that something was really challenging, if students spend a lot of time on one particular piece of content over another or if they found it useful. You’d have to contextualize that data based on what you were seeing in the course. But I think if you’re willing to look at that information, again, in the context of how your term went, it might just give you some passive information that could surprise you. I would even look at, for example, with assignment submissions, how many delays were there on certain assignments versus others? In which assignments did students request more extensions more than others? Again, this is just information that might help you inform whether the pacing was appropriate for the course, whether assignments were sequenced appropriately. That kind of thing. If you want to get more active, if you gave a pre-course survey, you can do a post-course survey. Most institutions, of course, have formal evaluations of teaching, but we know that institutional student evaluations of teaching can be fraught. Sometimes they ask the kinds of questions we don’t always want to ask or find most useful as instructors. So if you do your own very brief post evaluation, you could focus it on the design of the course itself. I think it’s worth asking students at the end of the course, how easy was the course site to navigate? How accessible did the materials feel for your ability to learn? You could return to some questions from your pre-course survey. If you asked a Likert scale about rating your confidence with learning something on a scale of one to 10 at the start of the course, you could ask them by the end, “How does your ranking change?” Even referring back to the original data that they might have submitted to you with the pre-course survey. So those are another way to ask them. I think if it’s possible to, what I love to at the end of the course is even a little brief post interview with students if possible. We mentioned this a bit in the book. Again, it’s time consuming. But if you have a small-ish class where you could have conferences at the end of the term, and have a moment with just a five-minute conversation to ask students: “How did it go? What aspects of the course design did you like most? Which were most challenging to you?” That’s another way to get information. Finally, I’ll just do one more technique we write about in the book, which is never discount your own reflection on your experience as well. This is another form of user research. Even though you are not the end user for the course, you are the designer, and so I think it’s always useful just to jot down a few notes and treat that as research when you’re done, too. What did you notice about user interactions on your course site throughout the term? What things surprised you? What things went exactly as you expected? You can use those notes to iterate and improve your experience for the next time that you offer it. So those are just a few techniques and many, which again, are drawn from the field of user experience research surveys, and interviews, for example, are pretty common user experience research practices… other UX research practices that, again, just depending on your time, depending on your resources, it’s great if you can see students engaging in the course as well, asking them, just really seeing what it looks like for them to interact in the course. That’s a good way to get at good information about it. I just want to encourage anyone who’s teaching not to shy away from getting that kind of feedback, because it does make, I think, teaching more satisfying when you’re getting more information about what’s working and what isn’t.

Rebecca: So, you know this question’s coming…[LAUGHTER] We always wrap up by asking: “What’s next?”

Jenae: Yes, I do know it’s coming, and it’s funny, because I was thinking about it. What am I [LAUGHTER] doing next? So to be honest, I don’t have a clearly defined project, I’m doing a lot of little things, I might be taking a little break, because I have written two books in about two and a half years [LAUGHTER]. So that’s been a lot… wonderful. I think I’ve been bitten by the writing bug, for sure. And so I suspect there’s more writing in my future, but nothing immediately next. I’m still very curious about what it’s going to mean to keep designing really good online learning experiences in the future, I don’t think we’re done with that conversation. I’m really curious about how that’s going to evolve in the context of creating more inclusive and equitable learning environments for students. So I imagine those are topics I will continue to explore to some extent, but we will see how, of course, with AI too, and the impacts of that on online learning, I’m sure there’s gonna be a whole set of ways to think about these topics that will continue to evolve. So I’m kind of keeping my eyes open and my ear to the ground on how things are developing. And we’ll just kind of see what ideas emerge from there.

Rebecca: Well, it’s always a pleasure to talk to you, Jenae. Thanks for all the work that you do.

Jenae: Likewise, thank you, again, for having me and for engaging with these excellent questions. And if you listened to this podcast, we’ll put in the speaker notes, I’ll give you a little gift of a promo code. If you’d like to buy the book, we can give you a 20% off discount with thanks to Rosenfeld Press who published this book.

John: Well thank you. We’ll be sure to include that in the show notes and it’s always great talking to you.

Jenae: Wonderful, and likewise, thank you again.

[MUSIC]

John: If you’ve enjoyed this podcast, please subscribe and leave a review on iTunes or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page.

Rebecca: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer.

Ganesh: Editing assistance by Ganesh.

[MUSIC]